gitea

Git with a cup of tea, painless self-hosted git service Mirror for internal git.with.parts use https://git.with.parts

History

Jason Song 375fd15fbf Refactor indexer (#25174 ) Refactor `modules/indexer` to make it more maintainable. And it can be easier to support more features. I'm trying to solve some of issue searching, this is a precursor to making functional changes. Current supported engines and the index versions: \| engines \| issues \| code \| \| - \| - \| - \| \| db \| Just a wrapper for database queries, doesn't need version \| - \| \| bleve \| The version of index is 2 \| The version of index is 6 \| \| elasticsearch \| The old index has no version, will be treated as version 0 in this PR \| The version of index is 1 \| \| meilisearch \| The old index has no version, will be treated as version 0 in this PR \| - \| ## Changes ### Split Splited it into mutiple packages ```text indexer ├── internal │ ├── bleve │ ├── db │ ├── elasticsearch │ └── meilisearch ├── code │ ├── bleve │ ├── elasticsearch │ └── internal └── issues ├── bleve ├── db ├── elasticsearch ├── internal └── meilisearch ``` - `indexer/interanal`: Internal shared package for indexer. - `indexer/interanal/[engine]`: Internal shared package for each engine (bleve/db/elasticsearch/meilisearch). - `indexer/code`: Implementations for code indexer. - `indexer/code/internal`: Internal shared package for code indexer. - `indexer/code/[engine]`: Implementation via each engine for code indexer. - `indexer/issues`: Implementations for issues indexer. ### Deduplication - Combine `Init/Ping/Close` for code indexer and issues indexer. - ~Combine `issues.indexerHolder` and `code.wrappedIndexer` to `internal.IndexHolder`.~ Remove it, use dummy indexer instead when the indexer is not ready. - Duplicate two copies of creating ES clients. - Duplicate two copies of `indexerID()`. ### Enhancement - [x] Support index version for elasticsearch issues indexer, the old index without version will be treated as version 0. - [x] Fix spell of `elastic_search/ElasticSearch`, it should be `Elasticsearch`. - [x] Improve versioning of ES index. We don't need `Aliases`: - Gitea does't need aliases for "Zero Downtime" because it never delete old indexes. - The old code of issues indexer uses the orignal name to create issue index, so it's tricky to convert it to an alias. - [x] Support index version for meilisearch issues indexer, the old index without version will be treated as version 0. - [x] Do "ping" only when `Ping` has been called, don't ping periodically and cache the status. - [x] Support the context parameter whenever possible. - [x] Fix outdated example config. - [x] Give up the requeue logic of issues indexer: When indexing fails, call Ping to check if it was caused by the engine being unavailable, and only requeue the task if the engine is unavailable. - It is fragile and tricky, could cause data losing (It did happen when I was doing some tests for this PR). And it works for ES only. - Just always requeue the failed task, if it caused by bad data, it's a bug of Gitea which should be fixed. --------- Co-authored-by: Giteabot <teabot@gitea.io>		1 year ago
..
actions	Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634 )	1 year ago
activitypub	Add Chef package registry (#22554 )	2 years ago
analyze	Implement FSFE REUSE for golang files (#21840 )	2 years ago
assetfs	Use a general approach to access custom/static/builtin assets (#24022 )	2 years ago
auth	Use a general approach to access custom/static/builtin assets (#24022 )	2 years ago
avatar	Improve avatar uploading / resizing / compressing, remove Fomantic card module (#24653 )	2 years ago
base	Update octicons and use `octicon-file-directory-symlink` (#25453 )	1 year ago
cache	Update redis library to support redis v7 (#24114 )	2 years ago
charset	Update go tool dependencies, restructure lint targets (#24239 )	2 years ago
container	Implement FSFE REUSE for golang files (#21840 )	2 years ago
context	Refactor indexer (#25174 )	1 year ago
csv	Refactor locale number (#24134 )	2 years ago
doctor	Refactor path & config system (#25330 )	1 year ago
emoji	Fix unstable emoji sort (#22346 )	2 years ago
eventsource	Move `convert` package to services (#22264 )	2 years ago
generate	Implement FSFE REUSE for golang files (#21840 )	2 years ago
git	Fix index generation parallelly failure (#25235 )	1 year ago
gitgraph	Add context cache as a request level cache (#22294 )	2 years ago
graceful	Improve queue and logger context (#24924 )	1 year ago
hcaptcha	Consume hcaptcha and pwn deps (#22610 )	2 years ago
highlight	test_env: hardcode major go version in use (#23464 )	2 years ago
hostmatcher	Implement FSFE REUSE for golang files (#21840 )	2 years ago
html	Implement FSFE REUSE for golang files (#21840 )	2 years ago
httpcache	Use standard HTTP library to serve files (#24693 )	2 years ago
httplib	Use fetch to send requests to create issues/comments (#25258 )	1 year ago
indexer	Refactor indexer (#25174 )	1 year ago
issue/template	Allow issue templates to not render title (#22589 )	2 years ago
json	Update gitea-vet to check FSFE REUSE (#22004 )	2 years ago
label	Make label templates have consistent behavior and priority (#23749 )	2 years ago
lfs	Rewrite logger system (#24726 )	2 years ago
log	Improve logger Pause handling (#24946 )	1 year ago
markup	Refactor path & config system (#25330 )	1 year ago
mcaptcha	Implement FSFE REUSE for golang files (#21840 )	2 years ago
metrics	Use a separate admin page to show global stats, remove `actions` stat (#25062 )	1 year ago
migration	Scoped labels (#22585 )	2 years ago
mirror	Improve queue and logger context (#24924 )	1 year ago
nosql	Update redis library to support redis v7 (#24114 )	2 years ago
notification	Rename NotifyPullReviewRequest to NotifyPullRequestReviewRequest (#24988 )	1 year ago
options	Use a general approach to access custom/static/builtin assets (#24022 )	2 years ago
packages	Add CRAN package registry (#22331 )	2 years ago
paginator	Use more specific test methods (#24265 )	2 years ago
pprof	Implement FSFE REUSE for golang files (#21840 )	2 years ago
private	Use the type RefName for all the needed places and fix pull mirror sync bugs (#24634 )	1 year ago
process	Do not output "Trace" level logs from process manager by default (#24952 )	1 year ago
proxy	Use proxy for pull mirror (#22771 )	2 years ago
proxyprotocol	Implement FSFE REUSE for golang files (#21840 )	2 years ago
public	Use standard HTTP library to serve files (#24693 )	2 years ago
queue	Help to recover from corrupted levelqueue (#24912 )	1 year ago
recaptcha	Implement FSFE REUSE for golang files (#21840 )	2 years ago
references	Use correct captured group range when parsing cross-reference (#22672 )	2 years ago
regexplru	Implement FSFE REUSE for golang files (#21840 )	2 years ago
repository	Move some regexp out of functions (#25430 )	1 year ago
secret	Improve decryption failure message (#24573 )	2 years ago
session	Update redis library to support redis v7 (#24114 )	2 years ago
setting	Import additional secrets via file uri (#25408 )	1 year ago
sitemap	Fix sitemap (#22272 )	2 years ago
ssh	Refactor path & config system (#25330 )	1 year ago
storage	Fix all possible setting error related storages and added some tests (#23911 )	1 year ago
structs	Fix `Permission` in API returned repository struct (#25388 )	1 year ago
svg	Use inline SVG for built-in OAuth providers (#25171 )	1 year ago
sync	Implement FSFE REUSE for golang files (#21840 )	2 years ago
system	Implement FSFE REUSE for golang files (#21840 )	2 years ago
templates	Fix missing commit message body when the message has leading newlines (#25418 )	1 year ago
test	Refactor web package and context package (#25298 )	1 year ago
testlogger	Refactor path & config system (#25330 )	1 year ago
timeutil	Fix incorrect webhook time and use relative-time to display it (#24477 )	2 years ago
translation	Refactor INI package (first step) (#25024 )	1 year ago
turnstile	Add new captcha: cloudflare turnstile (#22369 )	2 years ago
typesniffer	Do not recognize text files as audio (#23355 )	2 years ago
updatechecker	Implement FSFE REUSE for golang files (#21840 )	2 years ago
upload	Implement FSFE REUSE for golang files (#21840 )	2 years ago
uri	Implement FSFE REUSE for golang files (#21840 )	2 years ago
user	Implement FSFE REUSE for golang files (#21840 )	2 years ago
util	Store and use seconds for timeline time comments (#25392 )	1 year ago
validation	Map OIDC groups to Orgs/Teams (#21441 )	2 years ago
web	Refactor web package and context package (#25298 )	1 year ago
webhook	New webhook trigger for receiving Pull Request review requests (#24481 )	2 years ago