3 Commits (25dc1556cd70b567a4920beb002a0addfbfd6ef2)

Author	SHA1	Message	Date
wxiaoguang	6f9c278559	Rewrite queue (#24505) ... # ⚠️ Breaking Many deprecated queue config options are removed (actually, they should have been removed in 1.18/1.19). If you see the fatal message when starting Gitea: "Please update your app.ini to remove deprecated config options", please follow the error messages to remove these options from your app.ini. Example: ``` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].ISSUE_INDEXER_QUEUE_TYPE`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [E] Removed queue option: `[indexer].UPDATE_BUFFER_LEN`. Use new options in `[queue.issue_indexer]` 2023/05/06 19:39:22 [F] Please update your app.ini to remove deprecated config options ``` Many options in `[queue]` are are dropped, including: `WRAP_IF_NECESSARY`, `MAX_ATTEMPTS`, `TIMEOUT`, `WORKERS`, `BLOCK_TIMEOUT`, `BOOST_TIMEOUT`, `BOOST_WORKERS`, they can be removed from app.ini. # The problem The old queue package has some legacy problems: * complexity: I doubt few people could tell how it works. * maintainability: Too many channels and mutex/cond are mixed together, too many different structs/interfaces depends each other. * stability: due to the complexity & maintainability, sometimes there are strange bugs and difficult to debug, and some code doesn't have test (indeed some code is difficult to test because a lot of things are mixed together). * general applicability: although it is called "queue", its behavior is not a well-known queue. * scalability: it doesn't seem easy to make it work with a cluster without breaking its behaviors. It came from some very old code to "avoid breaking", however, its technical debt is too heavy now. It's a good time to introduce a better "queue" package. # The new queue package It keeps using old config and concept as much as possible. * It only contains two major kinds of concepts: * The "base queue": channel, levelqueue, redis * They have the same abstraction, the same interface, and they are tested by the same testing code. * The "WokerPoolQueue", it uses the "base queue" to provide "worker pool" function, calls the "handler" to process the data in the base queue. * The new code doesn't do "PushBack" * Think about a queue with many workers, the "PushBack" can't guarantee the order for re-queued unhandled items, so in new code it just does "normal push" * The new code doesn't do "pause/resume" * The "pause/resume" was designed to handle some handler's failure: eg: document indexer (elasticsearch) is down * If a queue is paused for long time, either the producers blocks or the new items are dropped. * The new code doesn't do such "pause/resume" trick, it's not a common queue's behavior and it doesn't help much. * If there are unhandled items, the "push" function just blocks for a few seconds and then re-queue them and retry. * The new code doesn't do "worker booster" * Gitea's queue's handlers are light functions, the cost is only the go-routine, so it doesn't make sense to "boost" them. * The new code only use "max worker number" to limit the concurrent workers. * The new "Push" never blocks forever * Instead of creating more and more blocking goroutines, return an error is more friendly to the server and to the end user. There are more details in code comments: eg: the "Flush" problem, the strange "code.index" hanging problem, the "immediate" queue problem. Almost ready for review. TODO: * [x] add some necessary comments during review * [x] add some more tests if necessary * [x] update documents and config options * [x] test max worker / active worker * [x] re-run the CI tasks to see whether any test is flaky * [x] improve the `handleOldLengthConfiguration` to provide more friendly messages * [x] fine tune default config values (eg: length?) ## Code coverage: 	2 years ago
John Olheiser	bb25f85ce8	Refactor docs (#23752) ... This was intended to be a small followup for https://github.com/go-gitea/gitea/pull/23712, but...here we are. 1. Our docs currently use `slug` as the entire URL, which makes refactoring tricky (see https://github.com/go-gitea/gitea/pull/23712). Instead, this PR attempts to make future refactoring easier by using slugs as an extension of the section. (Hugo terminology) - What the above boils down to is this PR attempts to use directory organization as URL management. e.g. `usage/comparison.en-us.md` -> `en-us/usage/comparison/`, `usage/packages/overview.en-us.md` -> `en-us/usage/packages/overview/` - Technically we could even remove `slug`, as Hugo defaults to using filename, however at least with this PR it means `slug` only needs to be the name for the **current file** rather than an entire URL 2. This PR adds appropriate aliases (redirects) for pages, so anything on the internet that links to our docs should hopefully not break. 3. A minor nit I've had for a while, renaming `seek-help` to `support`. It's a minor thing, but `seek-help` has a strange connotation to it. 4. The commits are split such that you can review the first which is the "actual" change, and the second is added redirects so that the first doesn't break links elsewhere. --------- Signed-off-by: jolheiser <john.olheiser@gmail.com>	2 years ago
Lunny Xiao	e8433b7fe6	Restructure documentation. Now the documentation has installation, administration, usage, development, contributing the 5 main parts (#23629) ... - **Installation**: includes how to install Gitea and related other tools, also includes upgrade Gitea - **Administration**: includes how to configure Gitea, customize Gitea and manage Gitea instance out of Gitea admin UI - **Usage**: includes how to use Gitea's functionalities. A sub documentation is about packages, in future we could also include CI/CD and others. - **Development**: includes how to integrate with Gitea's API, how to develop new features within Gitea - **Contributing**: includes how to contribute code to Gitea repositories. After this is merged, I think we can have a sub-documentation of `Usage` part named `Actions` to describe how to use Gitea actions --------- Co-authored-by: John Olheiser <john.olheiser@gmail.com>	2 years ago
Patrick Schratz	af79677a44	Reformat docs (#13897) ... Co-authored-by: 6543 <6543@obermui.de> Co-authored-by: Lauris BH <lauris@nix.lv>	4 years ago
6543	e7b47c5215	Format files (#13698) ... * align "make help" * format * untouch build/generate-svg.js * untouch .eslintrc * combine editorconfig's * rm editorconfig Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <techknowlogick@gitea.io>	4 years ago
Lauris BH	3c45cf8494	Add detected file language to code search (#10256) ... Move langauge detection to separate module to be more reusable Add option to disable vendored file exclusion from file search Allways show all language stats for search	5 years ago
guillep2k	72f6d5c882	Restrict repository indexing by glob match (#7767) ... * Restrict repository indexing by file extension * Use REPO_EXTENSIONS_LIST_INCLUDE instead of REPO_EXTENSIONS_LIST_EXCLUDE and have a more flexible extension pattern * Corrected to pass lint gosimple * Add wildcard support to REPO_INDEXER_EXTENSIONS * This reverts commit 72a650c8e4. * Add wildcard support to REPO_INDEXER_EXTENSIONS (no make vendor) * Simplify isIndexable() for better clarity * Add gobwas/glob to vendors * manually set appengine new release * Implement better REPO_INDEXER_INCLUDE and REPO_INDEXER_EXCLUDE * Add unit and integration tests * Update app.ini.sample and reword config-cheat-sheet * Add doc page and correct app.ini.sample * Some polish on the doc * Simplify code as suggested by @lafriks	5 years ago