Git with a cup of tea, painless self-hosted git service Mirror for internal git.with.parts use https://git.with.parts
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
gitea/routers/web/user/home.go

841 lines
24 KiB

// Copyright 2014 The Gogs Authors. All rights reserved.
// Copyright 2019 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
package user
import (
10 years ago
"bytes"
"fmt"
"net/http"
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
"regexp"
"slices"
"sort"
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
"strconv"
"strings"
activities_model "code.gitea.io/gitea/models/activities"
asymkey_model "code.gitea.io/gitea/models/asymkey"
"code.gitea.io/gitea/models/db"
issues_model "code.gitea.io/gitea/models/issues"
"code.gitea.io/gitea/models/organization"
repo_model "code.gitea.io/gitea/models/repo"
"code.gitea.io/gitea/models/unit"
user_model "code.gitea.io/gitea/models/user"
"code.gitea.io/gitea/modules/base"
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
"code.gitea.io/gitea/modules/container"
"code.gitea.io/gitea/modules/context"
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
issue_indexer "code.gitea.io/gitea/modules/indexer/issues"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/markup"
"code.gitea.io/gitea/modules/markup/markdown"
"code.gitea.io/gitea/modules/setting"
"code.gitea.io/gitea/modules/util"
"code.gitea.io/gitea/routers/web/feed"
context_service "code.gitea.io/gitea/services/context"
issue_service "code.gitea.io/gitea/services/issue"
pull_service "code.gitea.io/gitea/services/pull"
"github.com/keybase/go-crypto/openpgp"
"github.com/keybase/go-crypto/openpgp/armor"
"xorm.io/builder"
)
const (
tplDashboard base.TplName = "user/dashboard/dashboard"
tplIssues base.TplName = "user/dashboard/issues"
tplMilestones base.TplName = "user/dashboard/milestones"
tplProfile base.TplName = "user/profile"
)
// getDashboardContextUser finds out which context user dashboard is being viewed as .
func getDashboardContextUser(ctx *context.Context) *user_model.User {
ctxUser := ctx.Doer
orgName := ctx.Params(":org")
if len(orgName) > 0 {
ctxUser = ctx.Org.Organization.AsUser()
ctx.Data["Teams"] = ctx.Org.Teams
}
ctx.Data["ContextUser"] = ctxUser
orgs, err := organization.GetUserOrgsList(ctx, ctx.Doer)
if err != nil {
ctx.ServerError("GetUserOrgsList", err)
return nil
}
ctx.Data["Orgs"] = orgs
return ctxUser
}
// Dashboard render the dashboard page
func Dashboard(ctx *context.Context) {
9 years ago
ctxUser := getDashboardContextUser(ctx)
if ctx.Written() {
return
}
var (
date = ctx.FormString("date")
page = ctx.FormInt("page")
)
// Make sure page number is at least 1. Will be posted to ctx.Data.
if page <= 1 {
page = 1
}
ctx.Data["Title"] = ctxUser.DisplayName() + " - " + ctx.Tr("dashboard")
ctx.Data["PageIsDashboard"] = true
ctx.Data["PageIsNews"] = true
cnt, _ := organization.GetOrganizationCount(ctx, ctxUser)
ctx.Data["UserOrgsCount"] = cnt
ctx.Data["MirrorsEnabled"] = setting.Mirror.Enabled
ctx.Data["Date"] = date
var uid int64
if ctxUser != nil {
uid = ctxUser.ID
}
ctx.PageData["dashboardRepoList"] = map[string]any{
"searchLimit": setting.UI.User.RepoPagingNum,
"uid": uid,
}
if setting.Service.EnableUserHeatmap {
data, err := activities_model.GetUserHeatmapDataByUserTeam(ctx, ctxUser, ctx.Org.Team, ctx.Doer)
if err != nil {
ctx.ServerError("GetUserHeatmapDataByUserTeam", err)
return
}
ctx.Data["HeatmapData"] = data
ctx.Data["HeatmapTotalContributions"] = activities_model.GetTotalContributionsInHeatmap(data)
}
feeds, count, err := activities_model.GetFeeds(ctx, activities_model.GetFeedsOptions{
RequestedUser: ctxUser,
RequestedTeam: ctx.Org.Team,
Actor: ctx.Doer,
IncludePrivate: true,
OnlyPerformedBy: false,
IncludeDeleted: false,
Date: ctx.FormString("date"),
ListOptions: db.ListOptions{
Page: page,
PageSize: setting.UI.FeedPagingNum,
},
})
if err != nil {
ctx.ServerError("GetFeeds", err)
return
}
ctx.Data["Feeds"] = feeds
pager := context.NewPagination(int(count), setting.UI.FeedPagingNum, page, 5)
pager.AddParam(ctx, "date", "Date")
ctx.Data["Page"] = pager
ctx.HTML(http.StatusOK, tplDashboard)
}
// Milestones render the user milestones page
func Milestones(ctx *context.Context) {
if unit.TypeIssues.UnitGlobalDisabled() && unit.TypePullRequests.UnitGlobalDisabled() {
log.Debug("Milestones overview page not available as both issues and pull requests are globally disabled")
ctx.Status(http.StatusNotFound)
return
}
ctx.Data["Title"] = ctx.Tr("milestones")
ctx.Data["PageIsMilestonesDashboard"] = true
ctxUser := getDashboardContextUser(ctx)
if ctx.Written() {
return
}
repoOpts := repo_model.SearchRepoOptions{
Actor: ctx.Doer,
OwnerID: ctxUser.ID,
Private: true,
AllPublic: false, // Include also all public repositories of users and public organisations
AllLimited: false, // Include also all public repositories of limited organisations
Archived: util.OptionalBoolFalse,
HasMilestones: util.OptionalBoolTrue, // Just needs display repos has milestones
}
if ctxUser.IsOrganization() && ctx.Org.Team != nil {
repoOpts.TeamID = ctx.Org.Team.ID
}
var (
userRepoCond = repo_model.SearchRepositoryCondition(&repoOpts) // all repo condition user could visit
repoCond = userRepoCond
repoIDs []int64
reposQuery = ctx.FormString("repos")
isShowClosed = ctx.FormString("state") == "closed"
sortType = ctx.FormString("sort")
page = ctx.FormInt("page")
keyword = ctx.FormTrim("q")
)
if page <= 1 {
page = 1
}
if len(reposQuery) != 0 {
if issueReposQueryPattern.MatchString(reposQuery) {
// remove "[" and "]" from string
reposQuery = reposQuery[1 : len(reposQuery)-1]
// for each ID (delimiter ",") add to int to repoIDs
for _, rID := range strings.Split(reposQuery, ",") {
// Ensure nonempty string entries
if rID != "" && rID != "0" {
rIDint64, err := strconv.ParseInt(rID, 10, 64)
// If the repo id specified by query is not parseable or not accessible by user, just ignore it.
if err == nil {
repoIDs = append(repoIDs, rIDint64)
}
}
}
if len(repoIDs) > 0 {
// Don't just let repoCond = builder.In("id", repoIDs) because user may has no permission on repoIDs
// But the original repoCond has a limitation
repoCond = repoCond.And(builder.In("id", repoIDs))
}
} else {
log.Warn("issueReposQueryPattern not match with query")
}
}
counts, err := issues_model.CountMilestonesMap(ctx, issues_model.FindMilestoneOptions{
RepoCond: userRepoCond,
Name: keyword,
IsClosed: util.OptionalBoolOf(isShowClosed),
})
if err != nil {
ctx.ServerError("CountMilestonesByRepoIDs", err)
return
}
milestones, err := db.Find[issues_model.Milestone](ctx, issues_model.FindMilestoneOptions{
ListOptions: db.ListOptions{
Page: page,
PageSize: setting.UI.IssuePagingNum,
},
RepoCond: repoCond,
IsClosed: util.OptionalBoolOf(isShowClosed),
SortType: sortType,
Name: keyword,
})
if err != nil {
ctx.ServerError("SearchMilestones", err)
return
}
showRepos, _, err := repo_model.SearchRepositoryByCondition(ctx, &repoOpts, userRepoCond, false)
if err != nil {
ctx.ServerError("SearchRepositoryByCondition", err)
return
}
sort.Sort(showRepos)
for i := 0; i < len(milestones); {
for _, repo := range showRepos {
if milestones[i].RepoID == repo.ID {
milestones[i].Repo = repo
break
}
}
if milestones[i].Repo == nil {
log.Warn("Cannot find milestone %d 's repository %d", milestones[i].ID, milestones[i].RepoID)
milestones = append(milestones[:i], milestones[i+1:]...)
continue
}
milestones[i].RenderedContent, err = markdown.RenderString(&markup.RenderContext{
Links: markup.Links{
Base: milestones[i].Repo.Link(),
},
Metas: milestones[i].Repo.ComposeMetas(ctx),
Ctx: ctx,
}, milestones[i].Content)
if err != nil {
ctx.ServerError("RenderString", err)
return
}
if milestones[i].Repo.IsTimetrackerEnabled(ctx) {
err := milestones[i].LoadTotalTrackedTime(ctx)
if err != nil {
ctx.ServerError("LoadTotalTrackedTime", err)
return
}
}
i++
}
milestoneStats, err := issues_model.GetMilestonesStatsByRepoCondAndKw(ctx, repoCond, keyword)
if err != nil {
ctx.ServerError("GetMilestoneStats", err)
return
}
var totalMilestoneStats *issues_model.MilestonesStats
if len(repoIDs) == 0 {
totalMilestoneStats = milestoneStats
} else {
totalMilestoneStats, err = issues_model.GetMilestonesStatsByRepoCondAndKw(ctx, userRepoCond, keyword)
if err != nil {
ctx.ServerError("GetMilestoneStats", err)
return
}
}
showRepoIds := make(container.Set[int64], len(showRepos))
for _, repo := range showRepos {
if repo.ID > 0 {
showRepoIds.Add(repo.ID)
}
}
if len(repoIDs) == 0 {
repoIDs = showRepoIds.Values()
}
repoIDs = slices.DeleteFunc(repoIDs, func(v int64) bool {
return !showRepoIds.Contains(v)
})
var pagerCount int
if isShowClosed {
ctx.Data["State"] = "closed"
ctx.Data["Total"] = totalMilestoneStats.ClosedCount
pagerCount = int(milestoneStats.ClosedCount)
} else {
ctx.Data["State"] = "open"
ctx.Data["Total"] = totalMilestoneStats.OpenCount
pagerCount = int(milestoneStats.OpenCount)
}
ctx.Data["Milestones"] = milestones
ctx.Data["Repos"] = showRepos
ctx.Data["Counts"] = counts
ctx.Data["MilestoneStats"] = milestoneStats
ctx.Data["SortType"] = sortType
ctx.Data["Keyword"] = keyword
ctx.Data["RepoIDs"] = repoIDs
ctx.Data["IsShowClosed"] = isShowClosed
pager := context.NewPagination(pagerCount, setting.UI.IssuePagingNum, page, 5)
pager.AddParam(ctx, "q", "Keyword")
pager.AddParam(ctx, "repos", "RepoIDs")
pager.AddParam(ctx, "sort", "SortType")
pager.AddParam(ctx, "state", "State")
ctx.Data["Page"] = pager
ctx.HTML(http.StatusOK, tplMilestones)
}
// Pulls renders the user's pull request overview page
func Pulls(ctx *context.Context) {
if unit.TypePullRequests.UnitGlobalDisabled() {
log.Debug("Pull request overview page not available as it is globally disabled.")
ctx.Status(http.StatusNotFound)
return
}
ctx.Data["Title"] = ctx.Tr("pull_requests")
ctx.Data["PageIsPulls"] = true
buildIssueOverview(ctx, unit.TypePullRequests)
}
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
// Issues renders the user's issues overview page
func Issues(ctx *context.Context) {
if unit.TypeIssues.UnitGlobalDisabled() {
log.Debug("Issues overview page not available as it is globally disabled.")
ctx.Status(http.StatusNotFound)
return
}
ctx.Data["Title"] = ctx.Tr("issues")
ctx.Data["PageIsIssues"] = true
buildIssueOverview(ctx, unit.TypeIssues)
}
// Regexp for repos query
var issueReposQueryPattern = regexp.MustCompile(`^\[\d+(,\d+)*,?\]$`)
func buildIssueOverview(ctx *context.Context, unitType unit.Type) {
// ----------------------------------------------------
// Determine user; can be either user or organization.
// Return with NotFound or ServerError if unsuccessful.
// ----------------------------------------------------
ctxUser := getDashboardContextUser(ctx)
if ctx.Written() {
return
}
var (
viewType string
sortType = ctx.FormString("sort")
filterMode int
)
// Default to recently updated, unlike repository issues list
if sortType == "" {
sortType = "recentupdate"
}
// --------------------------------------------------------------------------------
// Distinguish User from Organization.
// Org:
// - Remember pre-determined viewType string for later. Will be posted to ctx.Data.
// Organization does not have view type and filter mode.
// User:
// - Use ctx.FormString("type") to determine filterMode.
// The type is set when clicking for example "assigned to me" on the overview page.
// - Remember either this or a fallback. Will be posted to ctx.Data.
// --------------------------------------------------------------------------------
// TODO: distinguish during routing
viewType = ctx.FormString("type")
switch viewType {
case "assigned":
filterMode = issues_model.FilterModeAssign
case "created_by":
filterMode = issues_model.FilterModeCreate
case "mentioned":
filterMode = issues_model.FilterModeMention
case "review_requested":
filterMode = issues_model.FilterModeReviewRequested
case "reviewed_by":
filterMode = issues_model.FilterModeReviewed
case "your_repositories":
fallthrough
default:
filterMode = issues_model.FilterModeYourRepositories
viewType = "your_repositories"
}
// --------------------------------------------------------------------------
// Build opts (IssuesOptions), which contains filter information.
// Will eventually be used to retrieve issues relevant for the overview page.
// Note: Non-final states of opts are used in-between, namely for:
// - Keyword search
// - Count Issues by repo
// --------------------------------------------------------------------------
// Get repository IDs where User/Org/Team has access.
var team *organization.Team
var org *organization.Organization
if ctx.Org != nil {
org = ctx.Org.Organization
team = ctx.Org.Team
}
isPullList := unitType == unit.TypePullRequests
opts := &issues_model.IssuesOptions{
IsPull: util.OptionalBoolOf(isPullList),
SortType: sortType,
IsArchived: util.OptionalBoolFalse,
Org: org,
Team: team,
User: ctx.Doer,
}
// Search all repositories which
//
// As user:
// - Owns the repository.
// - Have collaborator permissions in repository.
//
// As org:
// - Owns the repository.
//
// As team:
// - Team org's owns the repository.
// - Team has read permission to repository.
repoOpts := &repo_model.SearchRepoOptions{
Actor: ctx.Doer,
OwnerID: ctxUser.ID,
Private: true,
AllPublic: false,
AllLimited: false,
Collaborate: util.OptionalBoolNone,
UnitType: unitType,
Archived: util.OptionalBoolFalse,
}
if team != nil {
repoOpts.TeamID = team.ID
}
accessibleRepos := container.Set[int64]{}
{
ids, _, err := repo_model.SearchRepositoryIDs(ctx, repoOpts)
if err != nil {
ctx.ServerError("SearchRepositoryIDs", err)
return
}
accessibleRepos.AddMultiple(ids...)
opts.RepoIDs = ids
if len(opts.RepoIDs) == 0 {
// no repos found, don't let the indexer return all repos
opts.RepoIDs = []int64{0}
}
}
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
if ctx.Doer.ID == ctxUser.ID && filterMode != issues_model.FilterModeYourRepositories {
// If the doer is the same as the context user, which means the doer is viewing his own dashboard,
// it's not enough to show the repos that the doer owns or has been explicitly granted access to,
// because the doer may create issues or be mentioned in any public repo.
// So we need search issues in all public repos.
opts.AllPublic = true
}
switch filterMode {
case issues_model.FilterModeAll:
case issues_model.FilterModeYourRepositories:
case issues_model.FilterModeAssign:
opts.AssigneeID = ctx.Doer.ID
case issues_model.FilterModeCreate:
opts.PosterID = ctx.Doer.ID
case issues_model.FilterModeMention:
opts.MentionedID = ctx.Doer.ID
case issues_model.FilterModeReviewRequested:
opts.ReviewRequestedID = ctx.Doer.ID
case issues_model.FilterModeReviewed:
opts.ReviewedID = ctx.Doer.ID
}
// keyword holds the search term entered into the search field.
keyword := strings.Trim(ctx.FormString("q"), " ")
ctx.Data["Keyword"] = keyword
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
// Educated guess: Do or don't show closed issues.
isShowClosed := ctx.FormString("state") == "closed"
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
opts.IsClosed = util.OptionalBoolOf(isShowClosed)
// Make sure page number is at least 1. Will be posted to ctx.Data.
page := ctx.FormInt("page")
if page <= 1 {
page = 1
}
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
opts.Paginator = &db.ListOptions{
Page: page,
PageSize: setting.UI.IssuePagingNum,
}
// Get IDs for labels (a filter option for issues/pulls).
// Required for IssuesOptions.
var labelIDs []int64
selectedLabels := ctx.FormString("labels")
if len(selectedLabels) > 0 && selectedLabels != "0" {
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
var err error
labelIDs, err = base.StringsToInt64s(strings.Split(selectedLabels, ","))
if err != nil {
ctx.ServerError("StringsToInt64s", err)
return
}
}
opts.LabelIDs = labelIDs
// ------------------------------
// Get issues as defined by opts.
// ------------------------------
// Slice of Issues that will be displayed on the overview page
// USING FINAL STATE OF opts FOR A QUERY.
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
var issues issues_model.IssueList
{
issueIDs, _, err := issue_indexer.SearchIssues(ctx, issue_indexer.ToSearchOptions(keyword, opts))
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
if err != nil {
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
ctx.ServerError("issueIDsFromSearch", err)
return
}
issues, err = issues_model.GetIssuesByIDs(ctx, issueIDs, true)
if err != nil {
ctx.ServerError("GetIssuesByIDs", err)
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
return
}
}
commitStatuses, lastStatus, err := pull_service.GetIssuesAllCommitStatus(ctx, issues)
if err != nil {
ctx.ServerError("GetIssuesLastCommitStatus", err)
return
}
// -------------------------------
// Fill stats to post to ctx.Data.
// -------------------------------
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
issueStats, err := getUserIssueStats(ctx, ctxUser, filterMode, issue_indexer.ToSearchOptions(keyword, opts))
if err != nil {
ctx.ServerError("getUserIssueStats", err)
return
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
}
// Will be posted to ctx.Data.
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
var shownIssues int
if !isShowClosed {
shownIssues = int(issueStats.OpenCount)
} else {
shownIssues = int(issueStats.ClosedCount)
}
ctx.Data["IsShowClosed"] = isShowClosed
ctx.Data["IssueRefEndNames"], ctx.Data["IssueRefURLs"] = issue_service.GetRefEndNamesAndURLs(issues, ctx.FormString("RepoLink"))
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
if err := issues.LoadAttributes(ctx); err != nil {
ctx.ServerError("issues.LoadAttributes", err)
return
}
ctx.Data["Issues"] = issues
Refactor and enhance issue indexer to support both searching, filtering and paging (#26012) Fix #24662. Replace #24822 and #25708 (although it has been merged) ## Background In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete. To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer. ## Major changes - Redefine `IndexerData` to include all fields that could be used as filter conditions. - Refactor `Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string)` to `Search(ctx context.Context, options *SearchOptions)`, so it supports more conditions now. - Change the data type stored in `issueIndexerQueue`. Use `IndexerMetadata` instead of `IndexerData` in case the data has been updated while it is in the queue. This also reduces the storage size of the queue. - Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support `SearchOptions`. Also, update the data versions. - Keep most logic of database indexer, but remove `issues.SearchIssueIDsByKeyword` in `models` to avoid confusion where is the entry point to search issues. - Start a Meilisearch instance to test it in unit tests. - Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer. --------- Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
1 year ago
approvalCounts, err := issues.GetApprovalCounts(ctx)
if err != nil {
ctx.ServerError("ApprovalCounts", err)
return
}
ctx.Data["ApprovalCounts"] = func(issueID int64, typ string) int64 {
counts, ok := approvalCounts[issueID]
if !ok || len(counts) == 0 {
return 0
}
reviewTyp := issues_model.ReviewTypeApprove
if typ == "reject" {
reviewTyp = issues_model.ReviewTypeReject
} else if typ == "waiting" {
reviewTyp = issues_model.ReviewTypeRequest
}
for _, count := range counts {
if count.Type == reviewTyp {
return count.Count
}
}
return 0
}
ctx.Data["CommitLastStatus"] = lastStatus
ctx.Data["CommitStatuses"] = commitStatuses
ctx.Data["IssueStats"] = issueStats
ctx.Data["ViewType"] = viewType
ctx.Data["SortType"] = sortType
ctx.Data["IsShowClosed"] = isShowClosed
ctx.Data["SelectLabels"] = selectedLabels
if isShowClosed {
ctx.Data["State"] = "closed"
} else {
ctx.Data["State"] = "open"
}
[UI] IssuePage multi repo select (#8741) * Make repository list @ issues/PR page persist Will partially fix #6355 * Enable multiple selections in repo list @ issues/PR page Part of issue #6355 * Add RepoIDs to UserIssueStatsOptions to make "type" count correct when selecting one/multiple repos. * Replace variable "repo" with list "repos[]" and enable multiple selections of repositories from list by including/excluding RepoIDs in list. * * Remove redundant code * Add 'All' button Improves functionality of the page, so that backtracking is not necessary to reset the page * Remove redundant variable Completely replace 'RepoID' with 'RepoIDs' and remove redundant code * Add RepoIDs to label link * Revert part of code to previous version to troubleshoot build failure * Implement old and new pieces of code whilst adhering to multi select * Attempt to join the two versions Last commit passed tests but doesn't work in practice, this works in practice and hopefully passes the tests. * Update tests to desired state * Fix pagination implementation and tests * Pass repoIDs as `repos=[1,2,3...]` instead of several `repos[]=..` * Update tests file to reflect new functionality * Update template with new `repos` format * Implement new solution to show constant "total issues" count for "All" button * Correct behavior when passing zero to array * Comment out test url returning 404 This keeps returning 404 in the test despite working in practice, for the sake of running more tests I am commenting it out * Comment out another test url returning 404 Last attempt, if more tests crash I will uncomment the urls and request assistance. * Reenable tests and test fix * Re-enable tests * Make selecting "In your repositories" reset selection as passing IDs of repos belonging to other profiles causes breakage * Remove unnecessary (with multi-selection enable) code * Drop repo from repo map and total count if permission denied * Remove extra parenthesis * make template work again * find bug! * forgot the '#' at bugfixing * delete unused RepoID * compile regex only one time * make fmt * local variable = capital letter lower * check if repos query pattern is correct * pagination remove last , - make regex work again * use Replace instead of ReplaceAll; del delete * fix test * how did this test binary got in?!? dont forgot the "-p" at git add * ! * dont replace -> cut fisrt & last string Co-Authored-By: zeripath <art27@cantab.net> * jet another regex dont mind as long as it has the same result and is performatn ... Co-Authored-By: zeripath <art27@cantab.net> * dont use nonexisting repo for test * exclude /issues?type=created_by from test * add table to querys use same syntax in each query (table.colum) * add new issue for test * dont make a workaround or something else this need a refactor itself and is out of scope for this PR * fix misspell * CI.redo() * englisch txt update Co-Authored-By: zeripath <art27@cantab.net> * add sugestions * Tweak & Fix * CI.restart()
5 years ago
pager := context.NewPagination(shownIssues, setting.UI.IssuePagingNum, page, 5)
issue search on my related repositories (#9758) * adding search capability to user's issues dashboard * global issue search * placement of search bar on issues dashboard * fixed some bugs in the issue dashboard search * added unit test because IssueIDs option was added to UserIssueStatsOptions * some renaming of fields in the issue dashboard code to be more clear; also trying to fix issue of searching the right repos based on the filter * added unit test fro GetRepoIDsForIssuesOptions; fixed search lost on pagination; using shown issue status for open/close count; removed some debugging * fix issue with all count showing incorrectly * removed todo comment left in by mistake * typo pulling wrong count * fxied all count being off when selecting repositories * setting the opts.IsClosed after pulling repos to search, this is done so that the list of repo ids to serach for the keyword is not limited, we need to get all the issue ids for the shown issue stats * added "accessibleRepositoryCondition" check on the query to pull the repo ids to search for issues, this is an added protection to ensure we don't search repos the user does not have access to * added code so that in the issues search, we won't use an in clause of issues ids that goes over 1000 * fixed unit test * using 950 as the limit for issue search, removed unneeded group by in GetRepoIDsForIssuesOptions, showing search on pulls dashboard page too (not just issues) Co-authored-by: guillep2k <18600385+guillep2k@users.noreply.github.com>
5 years ago
pager.AddParam(ctx, "q", "Keyword")
pager.AddParam(ctx, "type", "ViewType")
pager.AddParam(ctx, "sort", "SortType")
pager.AddParam(ctx, "state", "State")
pager.AddParam(ctx, "labels", "SelectLabels")
pager.AddParam(ctx, "milestone", "MilestoneID")
pager.AddParam(ctx, "assignee", "AssigneeID")
ctx.Data["Page"] = pager
ctx.HTML(http.StatusOK, tplIssues)
}
8 years ago
// ShowSSHKeys output all the ssh keys of user by uid
func ShowSSHKeys(ctx *context.Context) {
keys, err := db.Find[asymkey_model.PublicKey](ctx, asymkey_model.FindPublicKeyOptions{
OwnerID: ctx.ContextUser.ID,
})
10 years ago
if err != nil {
ctx.ServerError("ListPublicKeys", err)
10 years ago
return
}
var buf bytes.Buffer
for i := range keys {
buf.WriteString(keys[i].OmitEmail())
buf.WriteString("\n")
10 years ago
}
ctx.PlainTextBytes(http.StatusOK, buf.Bytes())
10 years ago
}
// ShowGPGKeys output all the public GPG keys of user by uid
func ShowGPGKeys(ctx *context.Context) {
keys, err := db.Find[asymkey_model.GPGKey](ctx, asymkey_model.FindGPGKeyOptions{
ListOptions: db.ListOptionsAll,
OwnerID: ctx.ContextUser.ID,
})
if err != nil {
ctx.ServerError("ListGPGKeys", err)
return
}
entities := make([]*openpgp.Entity, 0)
failedEntitiesID := make([]string, 0)
for _, k := range keys {
e, err := asymkey_model.GPGKeyToEntity(ctx, k)
if err != nil {
if asymkey_model.IsErrGPGKeyImportNotExist(err) {
failedEntitiesID = append(failedEntitiesID, k.KeyID)
continue // Skip previous import without backup of imported armored key
}
ctx.ServerError("ShowGPGKeys", err)
return
}
entities = append(entities, e)
}
var buf bytes.Buffer
headers := make(map[string]string)
if len(failedEntitiesID) > 0 { // If some key need re-import to be exported
headers["Note"] = fmt.Sprintf("The keys with the following IDs couldn't be exported and need to be reuploaded %s", strings.Join(failedEntitiesID, ", "))
} else if len(entities) == 0 {
headers["Note"] = "This user hasn't uploaded any GPG keys."
}
writer, _ := armor.Encode(&buf, "PGP PUBLIC KEY BLOCK", headers)
for _, e := range entities {
err = e.Serialize(writer) // TODO find why key are exported with a different cipherTypeByte as original (should not be blocking but strange)
if err != nil {
ctx.ServerError("ShowGPGKeys", err)
return
}
}
writer.Close()
ctx.PlainTextBytes(http.StatusOK, buf.Bytes())
}
func UsernameSubRoute(ctx *context.Context) {
// WORKAROUND to support usernames with "." in it
// https://github.com/go-chi/chi/issues/781
username := ctx.Params("username")
reloadParam := func(suffix string) (success bool) {
ctx.SetParams("username", strings.TrimSuffix(username, suffix))
context_service.UserAssignmentWeb()(ctx)
// check view permissions
if !user_model.IsUserVisibleToViewer(ctx, ctx.ContextUser, ctx.Doer) {
ctx.NotFound("user", fmt.Errorf(ctx.ContextUser.Name))
return false
}
return !ctx.Written()
}
switch {
case strings.HasSuffix(username, ".png"):
if reloadParam(".png") {
AvatarByUserName(ctx)
}
case strings.HasSuffix(username, ".keys"):
if reloadParam(".keys") {
ShowSSHKeys(ctx)
}
case strings.HasSuffix(username, ".gpg"):
if reloadParam(".gpg") {
ShowGPGKeys(ctx)
}
case strings.HasSuffix(username, ".rss"):
if !setting.Other.EnableFeed {
ctx.Error(http.StatusNotFound)
return
}
if reloadParam(".rss") {
context_service.UserAssignmentWeb()(ctx)
feed.ShowUserFeedRSS(ctx)
}
case strings.HasSuffix(username, ".atom"):
if !setting.Other.EnableFeed {
ctx.Error(http.StatusNotFound)
return
}
if reloadParam(".atom") {
feed.ShowUserFeedAtom(ctx)
}
default:
context_service.UserAssignmentWeb()(ctx)
if !ctx.Written() {
ctx.Data["EnableFeed"] = setting.Other.EnableFeed
Fix inconsistent user profile layout across tabs (#25625) Fix ::User Profile Page Project Tab Have Inconsistent Layout and Style Added the big_avator for consistency in the all header_items tabs. Fixes: #24871 > ### Description > in the user profile page the `Packages` and `Projects` tab have small icons for user but other tabs have bigger profile picture with user info: > > ### Screenshots > ### **For Packages And Projects:** > ![image](https://user-images.githubusercontent.com/25511175/240148601-2420d77b-ba25-4718-9ccb-c5d0d95e3079.png) > > ### **For Other Tabs:** > ![image](https://user-images.githubusercontent.com/25511175/240148461-ce9636b3-fe11-4c46-a230-30d83eee5947.png) > ## Before ![image](https://github.com/go-gitea/gitea/assets/80308335/975ad038-07ca-4b10-b75d-ccf259be7b9d) ## After changes Project View <img width="1394" alt="image" src="https://github.com/go-gitea/gitea/assets/80308335/95d181d7-8e61-496d-9899-7b825c91ad56"> Packages View <img width="1378" alt="image" src="https://github.com/go-gitea/gitea/assets/80308335/7f5fd60f-6b18-4fa8-8c56-7b0d45d1a610"> ## Org view for projects page <img width="1385" alt="image" src="https://github.com/go-gitea/gitea/assets/80308335/6400dc89-a5ae-4f0a-831b-5b6efa020d89"> ## Org view for packages page <img width="1387" alt="image" src="https://github.com/go-gitea/gitea/assets/80308335/4e1e9ffe-1e4b-4334-8657-de11b5fd31d0"> --------- Co-authored-by: wxiaoguang <wxiaoguang@gmail.com> Co-authored-by: Giteabot <teabot@gitea.io> Co-authored-by: silverwind <me@silverwind.io>
1 year ago
OwnerProfile(ctx)
}
}
}
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
func getUserIssueStats(ctx *context.Context, ctxUser *user_model.User, filterMode int, opts *issue_indexer.SearchOptions) (*issues_model.IssueStats, error) {
doerID := ctx.Doer.ID
opts = opts.Copy(func(o *issue_indexer.SearchOptions) {
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
// If the doer is the same as the context user, which means the doer is viewing his own dashboard,
// it's not enough to show the repos that the doer owns or has been explicitly granted access to,
// because the doer may create issues or be mentioned in any public repo.
// So we need search issues in all public repos.
o.AllPublic = doerID == ctxUser.ID
o.AssigneeID = nil
o.PosterID = nil
o.MentionID = nil
o.ReviewRequestedID = nil
o.ReviewedID = nil
})
var (
err error
ret = &issues_model.IssueStats{}
)
{
openClosedOpts := opts.Copy()
switch filterMode {
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
case issues_model.FilterModeAll:
// no-op
case issues_model.FilterModeYourRepositories:
openClosedOpts.AllPublic = false
case issues_model.FilterModeAssign:
openClosedOpts.AssigneeID = &doerID
case issues_model.FilterModeCreate:
openClosedOpts.PosterID = &doerID
case issues_model.FilterModeMention:
openClosedOpts.MentionID = &doerID
case issues_model.FilterModeReviewRequested:
openClosedOpts.ReviewRequestedID = &doerID
case issues_model.FilterModeReviewed:
openClosedOpts.ReviewedID = &doerID
}
openClosedOpts.IsClosed = util.OptionalBoolFalse
ret.OpenCount, err = issue_indexer.CountIssues(ctx, openClosedOpts)
if err != nil {
return nil, err
}
openClosedOpts.IsClosed = util.OptionalBoolTrue
ret.ClosedCount, err = issue_indexer.CountIssues(ctx, openClosedOpts)
if err != nil {
return nil, err
}
}
Include public repos in doer's dashboard for issue search (#28304) It will fix #28268 . <img width="1313" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/cb1e07d5-7a12-4691-a054-8278ba255bfc"> <img width="1318" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/4fd60820-97f1-4c2c-a233-d3671a5039e9"> ## :warning: BREAKING :warning: But need to give up some features: <img width="1312" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/281c0d51-0e7d-473f-bbed-216e2f645610"> However, such abandonment may fix #28055 . ## Backgroud When the user switches the dashboard context to an org, it means they want to search issues in the repos that belong to the org. However, when they switch to themselves, it means all repos they can access because they may have created an issue in a public repo that they don't own. <img width="286" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/182dcd5b-1c20-4725-93af-96e8dfae5b97"> It's a confusing design. Think about this: What does "In your repositories" mean when the user switches to an org? Repos belong to the user or the org? Whatever, it has been broken by #26012 and its following PRs. After the PR, it searches for issues in repos that the dashboard context user owns or has been explicitly granted access to, so it causes #28268. ## How to fix it It's not really difficult to fix it. Just extend the repo scope to search issues when the dashboard context user is the doer. Since the user may create issues or be mentioned in any public repo, we can just set `AllPublic` to true, which is already supported by indexers. The DB condition will also support it in this PR. But the real difficulty is how to count the search results grouped by repos. It's something like "search issues with this keyword and those filters, and return the total number and the top results. **Then, group all of them by repo and return the counts of each group.**" <img width="314" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/5206eb20-f8f5-49b9-b45a-1be2fcf679f4"> Before #26012, it was being done in the DB, but it caused the results to be incomplete (see the description of #26012). And to keep this, #26012 implement it in an inefficient way, just count the issues by repo one by one, so it cannot work when `AllPublic` is true because it's almost impossible to do this for all public repos. https://github.com/go-gitea/gitea/blob/1bfcdeef4cca0f5509476358e5931c13d37ed1ca/modules/indexer/issues/indexer.go#L318-L338 ## Give up unnecessary features We may can resovle `TODO: use "group by" of the indexer engines to implement it`, I'm sure it can be done with Elasticsearch, but IIRC, Bleve and Meilisearch don't support "group by". And the real question is, does it worth it? Why should we need to know the counts grouped by repos? Let me show you my search dashboard on gitea.com. <img width="1304" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/2bca2d46-6c71-4de1-94cb-0c9af27c62ff"> I never think the long repo list helps anything. And if we agree to abandon it, things will be much easier. That is this PR. ## TODO I know it's important to filter by repos when searching issues. However, it shouldn't be the way we have it now. It could be implemented like this. <img width="1316" alt="image" src="https://github.com/go-gitea/gitea/assets/9418365/99ee5f21-cbb5-4dfe-914d-cb796cb79fbe"> The indexers support it well now, but it requires some frontend work, which I'm not good at. So, I think someone could help do that in another PR and merge this one to fix the bug first. Or please block this PR and help to complete it. Finally, "Switch dashboard context" is also a design that needs improvement. In my opinion, it can be accomplished by adding filtering conditions instead of "switching".
12 months ago
ret.YourRepositoriesCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.AllPublic = false }))
if err != nil {
return nil, err
}
ret.AssignCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.AssigneeID = &doerID }))
if err != nil {
return nil, err
}
ret.CreateCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.PosterID = &doerID }))
if err != nil {
return nil, err
}
ret.MentionCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.MentionID = &doerID }))
if err != nil {
return nil, err
}
ret.ReviewRequestedCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.ReviewRequestedID = &doerID }))
if err != nil {
return nil, err
}
ret.ReviewedCount, err = issue_indexer.CountIssues(ctx, opts.Copy(func(o *issue_indexer.SearchOptions) { o.ReviewedID = &doerID }))
if err != nil {
return nil, err
}
return ret, nil
}