6 Commits (30668e0047d684feec83b0e316c001c6e2332986)

Author	SHA1	Message	Date
Lunny Xiao	bd820aa9c5	Add context cache as a request level cache (#22294) ... To avoid duplicated load of the same data in an HTTP request, we can set a context cache to do that. i.e. Some pages may load a user from a database with the same id in different areas on the same page. But the code is hidden in two different deep logic. How should we share the user? As a result of this PR, now if both entry functions accept `context.Context` as the first parameter and we just need to refactor `GetUserByID` to reuse the user from the context cache. Then it will not be loaded twice on an HTTP request. But of course, sometimes we would like to reload an object from the database, that's why `RemoveContextData` is also exposed. The core context cache is here. It defines a new context ```go type cacheContext struct { ctx context.Context data map[any]map[any]any lock sync.RWMutex } var cacheContextKey = struct{}{} func WithCacheContext(ctx context.Context) context.Context { return context.WithValue(ctx, cacheContextKey, &cacheContext{ ctx: ctx, data: make(map[any]map[any]any), }) } ``` Then you can use the below 4 methods to read/write/del the data within the same context. ```go func GetContextData(ctx context.Context, tp, key any) any func SetContextData(ctx context.Context, tp, key, value any) func RemoveContextData(ctx context.Context, tp, key any) func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error) ``` Then let's take a look at how `system.GetString` implement it. ```go func GetSetting(ctx context.Context, key string) (string, error) { return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) { return cache.GetString(genSettingCacheKey(key), func() (string, error) { res, err := GetSettingNoCache(ctx, key) if err != nil { return "", err } return res.SettingValue, nil }) }) } ``` First, it will check if context data include the setting object with the key. If not, it will query from the global cache which may be memory or a Redis cache. If not, it will get the object from the database. In the end, if the object gets from the global cache or database, it will be set into the context cache. An object stored in the context cache will only be destroyed after the context disappeared.	2 years ago
flynnnnnnnnnn	e81ccc406b	Implement FSFE REUSE for golang files (#21840) ... Change all license headers to comply with REUSE specification. Fix #16132 Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github> Co-authored-by: John Olheiser <john.olheiser@gmail.com>	2 years ago
delvh	0ebb45cfe7	Replace all instances of fmt.Errorf(%v) with fmt.Errorf(%w) (#21551) ... Found using `find . -type f -name '*.go' -print -exec vim {} -c ':%s/fmt\.Errorf(\(.*\)%v\(.*\)err/fmt.Errorf(\1%w\2err/g' -c ':wq' \;` Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Andrew Thornton <art27@cantab.net> Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>	2 years ago
John Olheiser	a48d6ba4b4	Go 1.19 format (#20758) ... * 1.19 gofumpt Signed-off-by: jolheiser <john.olheiser@gmail.com> * Change CSV test Signed-off-by: jolheiser <john.olheiser@gmail.com> * Commit whitespace fixes from @zeripath Co-authored-by: zeripath <art27@cantab.net> * Update emoji Signed-off-by: jolheiser <john.olheiser@gmail.com> * bump swagger & fix generate-swagger Signed-off-by: jolheiser <john.olheiser@gmail.com> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>	2 years ago
Wim	cb50375e2b	Add more linters to improve code readability (#19989) ... Add nakedret, unconvert, wastedassign, stylecheck and nolintlint linters to improve code readability - nakedret - https://github.com/alexkohler/nakedret - nakedret is a Go static analysis tool to find naked returns in functions greater than a specified function length. - unconvert - https://github.com/mdempsky/unconvert - Remove unnecessary type conversions - wastedassign - https://github.com/sanposhiho/wastedassign - wastedassign finds wasted assignment statements. - notlintlint - Reports ill-formed or insufficient nolint directives - stylecheck - https://staticcheck.io/docs/checks/#ST - keep style consistent - excluded: [ST1003 - Poorly chosen identifier](https://staticcheck.io/docs/checks/#ST1003) and [ST1005 - Incorrectly formatted error string](https://staticcheck.io/docs/checks/#ST1005)	2 years ago
Anthony Wang	e86f18a05a	User keypairs and HTTP signatures for ActivityPub federation using go-ap (#19133) ... * go.mod: add go-fed/{httpsig,activity/pub,activity/streams} dependency go get github.com/go-fed/activity/streams@master go get github.com/go-fed/activity/pub@master go get github.com/go-fed/httpsig@master * activitypub: implement /api/v1/activitypub/user/{username} (#14186) Return informations regarding a Person (as defined in ActivityStreams https://www.w3.org/TR/activitystreams-vocabulary/#dfn-person). Refs: https://github.com/go-gitea/gitea/issues/14186 Signed-off-by: Loïc Dachary <loic@dachary.org> * activitypub: add the public key to Person (#14186) Refs: https://github.com/go-gitea/gitea/issues/14186 Signed-off-by: Loïc Dachary <loic@dachary.org> * activitypub: go-fed conformant Clock instance Signed-off-by: Loïc Dachary <loic@dachary.org> * activitypub: signing http client Signed-off-by: Loïc Dachary <loic@dachary.org> * activitypub: implement the ReqSignature middleware Signed-off-by: Loïc Dachary <loic@dachary.org> * activitypub: hack_16834 Signed-off-by: Loïc Dachary <loic@dachary.org> * Fix CI checks-backend errors with go mod tidy Signed-off-by: Anthony Wang <ta180m@pm.me> * Change 2021 to 2022, properly format package imports Signed-off-by: Anthony Wang <ta180m@pm.me> * Run make fmt and make generate-swagger Signed-off-by: Anthony Wang <ta180m@pm.me> * Use Gitea JSON library, add assert for pkp Signed-off-by: Anthony Wang <ta180m@pm.me> * Run make fmt again, fix err var redeclaration Signed-off-by: Anthony Wang <ta180m@pm.me> * Remove LogSQL from ActivityPub person test Signed-off-by: Anthony Wang <ta180m@pm.me> * Assert if json.Unmarshal succeeds Signed-off-by: Anthony Wang <ta180m@pm.me> * Cleanup, handle invalid usernames for ActivityPub person GET request Signed-off-by: Anthony Wang <ta180m@pm.me> * Rename hack_16834 to user_settings Signed-off-by: Anthony Wang <ta180m@pm.me> * Use the httplib module instead of http for GET requests * Clean up whitespace with make fmt * Use time.RFC1123 and make the http.Client proxy-aware * Check if digest algo is supported in setting module * Clean up some variable declarations * Remove unneeded copy * Use system timezone instead of setting.DefaultUILocation * Use named constant for httpsigExpirationTime * Make pubKey IRI #main-key instead of /#main-key * Move /#main-key to #main-key in tests * Implemented Webfinger endpoint. * Add visible check. * Add user profile as alias. * Add actor IRI and remote interaction URL to WebFinger response * fmt * Fix lint errors * Use go-ap instead of go-fed * Run go mod tidy to fix missing modules in go.mod and go.sum * make fmt * Convert remaining code to go-ap * Clean up go.sum * Fix JSON unmarshall error * Fix CI errors by adding @context to Person() and making sure types match * Correctly decode JSON in api_activitypub_person_test.go * Force CI rerun * Fix TestActivityPubPersonInbox segfault * Fix lint error * Use @mariusor's suggestions for idiomatic go-ap usage * Correctly add inbox/outbox IRIs to person * Code cleanup * Remove another LogSQL from ActivityPub person test * Move httpsig algos slice to an init() function * Add actor IRI and remote interaction URL to WebFinger response * Update TestWebFinger to check for ActivityPub IRI in aliases * make fmt * Force CI rerun * WebFinger: Add CORS header and fix Href -> Template for remote interactions The CORS header is needed due to https://datatracker.ietf.org/doc/html/rfc7033#section-5 and fixes some Peertube <-> Gitea federation issues * make lint-backend * Make sure Person endpoint has Content-Type application/activity+json and includes PreferredUsername, URL, and Icon Setting the correct Content-Type is essential for federating with Mastodon * Use UTC instead of GMT * Rename pkey to pubKey * Make sure HTTP request Date in GMT * make fmt * dont drop err * Make sure API responses always refer to username in original case Copied from what I wrote on #19133 discussion: Handling username case is a very tricky issue and I've already encountered a Mastodon <-> Gitea federation bug due to Gitea considering Ta180m and ta180m to be the same user while Mastodon thinks they are two different users. I think the best way forward is for Gitea to only use the original case version of the username for federation so other AP software don't get confused. * Move httpsig algs constant slice to modules/setting/federation.go * Add new federation settings to app.example.ini and config-cheat-sheet * Return if marshalling error * Make sure Person IRIs are generated correctly This commit ensures that if the setting.AppURL is something like "http://127.0.0.1:42567" (like in the integration tests), a trailing slash will be added after that URL. * If httpsig verification fails, fix Host header and try again This fixes a very rare bug when Gitea and another AP server (confirmed to happen with Mastodon) are running on the same machine, Gitea fails to verify incoming HTTP signatures. This is because the other AP server creates the sig with the public Gitea domain as the Host. However, when Gitea receives the request, the Host header is instead localhost, so the signature verification fails. Manually changing the host header to the correct value and trying the veification again fixes the bug. * Revert "If httpsig verification fails, fix Host header and try again" This reverts commit f53e46c721. The bug was actually caused by nginx messing up the Host header when reverse-proxying since I didn't have the line `proxy_set_header Host $host;` in my nginx config for Gitea. * Go back to using ap.IRI to generate inbox and outbox IRIs * use const for key values * Update routers/web/webfinger.go * Use ctx.JSON in Person response to make code cleaner * Revert "Use ctx.JSON in Person response to make code cleaner" This doesn't work because the ctx.JSON() function already sends the response out and it's too late to edit the headers. This reverts commit 95aad98897. * Use activitypub.ActivityStreamsContentType for Person response Content Type * Limit maximum ActivityPub request and response sizes to a configurable setting * Move setting key constants to models/user/setting_keys.go * Fix failing ActivityPubPerson integration test by checking the correct field for username * Add a warning about changing settings that can break federation * Add better comments * Don't multiply Federation.MaxSize by 1<<20 twice * Add more better comments * Fix failing ActivityPubMissingPerson test We now use ctx.ContextUser so the message printed out when a user does not exist is slightly different * make generate-swagger For some reason I didn't realize that /templates/swagger/v1_json.tmpl was machine-generated by make generate-swagger... I've been editing it by hand for three months! 🤦 * Move getting the RFC 2616 time to a separate function * More code cleanup * Update go-ap to fix empty liked collection and removed unneeded HTTP headers * go mod tidy * Add ed25519 to httpsig algorithms * Use go-ap/jsonld to add @context and marshal JSON * Change Gitea user agent from the default to Gitea/Version * Use ctx.ServerError and remove all remote interaction code from webfinger.go	2 years ago