This PR follows #21535 (and replace #22592)
## Review without space diff
https://github.com/go-gitea/gitea/pull/22678/files?diff=split&w=1
## Purpose of this PR
1. Make git module command completely safe (risky user inputs won't be
passed as argument option anymore)
2. Avoid low-level mistakes like
https://github.com/go-gitea/gitea/pull/22098#discussion_r1045234918
3. Remove deprecated and dirty `CmdArgCheck` function, hide the `CmdArg`
type
4. Simplify code when using git command
## The main idea of this PR
* Move the `git.CmdArg` to the `internal` package, then no other package
except `git` could use it. Then developers could never do
`AddArguments(git.CmdArg(userInput))` any more.
* Introduce `git.ToTrustedCmdArgs`, it's for user-provided and already
trusted arguments. It's only used in a few cases, for example: use git
arguments from config file, help unit test with some arguments.
* Introduce `AddOptionValues` and `AddOptionFormat`, they make code more
clear and simple:
* Before: `AddArguments("-m").AddDynamicArguments(message)`
* After: `AddOptionValues("-m", message)`
* -
* Before: `AddArguments(git.CmdArg(fmt.Sprintf("--author='%s <%s>'",
sig.Name, sig.Email)))`
* After: `AddOptionFormat("--author='%s <%s>'", sig.Name, sig.Email)`
## FAQ
### Why these changes were not done in #21535 ?
#21535 is mainly a search&replace, it did its best to not change too
much logic.
Making the framework better needs a lot of changes, so this separate PR
is needed as the second step.
### The naming of `AddOptionXxx`
According to git's manual, the `--xxx` part is called `option`.
### How can it guarantee that `internal.CmdArg` won't be not misused?
Go's specification guarantees that. Trying to access other package's
internal package causes compilation error.
And, `golangci-lint` also denies the git/internal package. Only the
`git/command.go` can use it carefully.
### There is still a `ToTrustedCmdArgs`, will it still allow developers
to make mistakes and pass untrusted arguments?
Generally speaking, no. Because when using `ToTrustedCmdArgs`, the code
will be very complex (see the changes for examples). Then developers and
reviewers can know that something might be unreasonable.
### Why there was a `CmdArgCheck` and why it's removed?
At the moment of #21535, to reduce unnecessary changes, `CmdArgCheck`
was introduced as a hacky patch. Now, almost all code could be written
as `cmd := NewCommand(); cmd.AddXxx(...)`, then there is no need for
`CmdArgCheck` anymore.
### Why many codes for `signArg == ""` is deleted?
Because in the old code, `signArg` could never be empty string, it's
either `-S[key-id]` or `--no-gpg-sign`. So the `signArg == ""` is just
dead code.
---------
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Change all license headers to comply with REUSE specification.
Fix#16132
Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
A testing cleanup.
This pull request replaces `os.MkdirTemp` with `t.TempDir`. We can use the `T.TempDir` function from the `testing` package to create temporary directory. The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete.
This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot.
Reference: https://pkg.go.dev/testing#T.TempDir
```go
func TestFoo(t *testing.T) {
// before
tmpDir, err := os.MkdirTemp("", "")
require.NoError(t, err)
defer os.RemoveAll(tmpDir)
// now
tmpDir := t.TempDir()
}
```
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Follows #19266, #8553, Close#18553, now there are only three `Run..(&RunOpts{})` functions.
* before: `stdout, err := RunInDir(path)`
* now: `stdout, _, err := RunStdString(&git.RunOpts{Dir:path})`
- Pass the Global command args into serviceRPC.
- Fixes error with partial cloning.
- Add partial clone test
- Include diff
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
This PR continues the work in #17125 by progressively ensuring that git
commands run within the request context.
This now means that the if there is a git repo already open in the context it will be used instead of reopening it.
Signed-off-by: Andrew Thornton <art27@cantab.net>
* Make test work with different default branch config
On a system configured with a different default branch name, doGitInitTestRepository
will create a repository with a branch named differently, and so further tests
would break:
--- FAIL: TestGit/HTTP/PushCreate/SuccessfullyPushAndCreateTestRepository (0.02s)
git_helper_for_declarative_test.go:167:
Error Trace: git_helper_for_declarative_test.go:167
Error: Received unexpected error:
exit status 1 - error: src refspec master does not match any
error: failed to push some refs to 'http://127.0.0.1:3003/user2/repo-tmp-push-create-http.git'
Test: TestGit/HTTP/PushCreate/SuccessfullyPushAndCreateTestRepository
git_test.go:587:
Error Trace: git_test.go:587
Error: Received unexpected error:
repository does not exist [id: 0, uid: 0, owner_name: user2, name: repo-tmp-push-create-http]
Test: TestGit/HTTP/PushCreate
* Update integrations/git_helper_for_declarative_test.go
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
* Benchmark Integration TESTS
* CI: add benching-arm64 pipeline
* BenchmarkRepo: name test case tests
* Fix BenchmarkRepoBranchCommit beside Create new Branch
* CI: benching use amd64
* rm total broken "BenchmarkRepo"
* dont run benchmark in CI
* remove github.com/unknwon/com from models
* dont use "com.ToStr()"
* replace "com.ToStr" with "fmt.Sprint" where its easy to do
* more refactor
* fix test
* just "proxy" Copy func for now
* as per @lunny
Replace all calls to os.Remove/os.RemoveAll by retrying util.Remove/util.RemoveAll and remove circular dependencies from util.
Fix#12339
Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: silverwind <me@silverwind.io>
Reduce the code duplication in the PushCreate test and switch
to a declarative format.
* Instead of explicitly creating the repository re-use functions from the other declarative tests and add comments
* Ensure that the test repository is deleted at the end of test
* Slightly reorder the sub-tests
Also reduce the code duplication in MergeFork and add some comments there too and make doGitCloneFail be self-contained.
Signed-off-by: Andrew Thornton art27@cantab.net
* Fix the intermittent TestGPGGit failures
Reattempt to open the listener if the port is busy with a delay up to a second
Switch from generating a private key each time, just use a known good key
* Yet another attempt to fix the race in gpg_git_test
* add some fail nows
* Need to set preparetestenv
* Ensure that http messages go to the correct server
* Change tests to make it possible to run TestGit with 1.7.2
* Make merge run on 1.7.2
* Fix tracking and staging branch name problem
* Ensure that git 1.7.2 works on tests
* ensure that there is no chance for conflicts
* Fix-up missing merge issues
* Final rm
* Ensure LFS filters run on the tests
* Do not sign commits from temp repo
* Restore tracking fetch change
* Apply suggestions from code review
* Update modules/repofiles/temp_repo.go
* Rename BaseLogger to WriterLogger to help the creation of other providers
* Don't export ColorBytes and ResetBytes from ColoredValues
* Make integration tests only print logs if they fail
* check can color before coloring
* I always forget about MSSQL
* Oh and use LEVEL in sqlite.ini
* Make the test logger log at info - as it means you see the router
* Remove empty expected changes
* Make the migrations quieter too
* Don't display SQL on error - it can be looked at in the file logs if necessary
* Fix skip when using onGiteaRun
* use native golang SSH library but ssh-keygen when enable built-in SSH server to remove dependent on that command lines
* fix tests and add comment head
1. A key can either be an ssh user key or a deploy key. It cannot be both.
2. If a key is a user key - it can only be associated with one user.
3. If a key is a deploy key - it can be used in multiple repositories and the permissions it has on those repositories can be different.
4. If a repository is deleted, its deploy keys must be deleted too.
We currently don't enforce any of this and multiple repositories access with different permissions doesn't work at all. This PR enforces the following constraints:
- [x] You should not be able to add the same user key as another user
- [x] You should not be able to add a ssh user key which is being used as a deploy key
- [x] You should not be able to add a ssh deploy key which is being used as a user key
- [x] If you add an ssh deploy key to another repository you should be able to use it in different modes without losing the ability to use it in the other mode.
- [x] If you delete a repository you must delete all its deploy keys.
Fix#1357
wxiaoguang
flynnnnnnnnnn
Eng Zer Jun
Kyle D
6543
Gusted
zeripath
mscherer
Wim
Lunny Xiao
Tamal Saha
guillep2k