mirror of https://github.com/go-gitea/gitea
Add Debian package registry (#24426)
Co-authored-by: @awkwardbunny This PR adds a Debian package registry. You can follow [this tutorial](https://www.baeldung.com/linux/create-debian-package) to build a *.deb package for testing. Source packages are not supported at the moment and I did not find documentation of the architecture "all" and how these packages should be treated.  Part of #20751. Revised copy of #22854. --------- Co-authored-by: Brian Hong <brian@hongs.me> Co-authored-by: techknowlogick <techknowlogick@gitea.io> Co-authored-by: Giteabot <teabot@gitea.io>pull/24478/head^2
KN4CK3R
2 years ago
committed by
GitHub
parent
1f52560ca4
commit
bf999e4069
@ -0,0 +1,134 @@ |
|||||||
|
--- |
||||||
|
date: "2023-01-07T00:00:00+00:00" |
||||||
|
title: "Debian Packages Repository" |
||||||
|
slug: "packages/debian" |
||||||
|
draft: false |
||||||
|
toc: false |
||||||
|
menu: |
||||||
|
sidebar: |
||||||
|
parent: "packages" |
||||||
|
name: "Debian" |
||||||
|
weight: 35 |
||||||
|
identifier: "debian" |
||||||
|
--- |
||||||
|
|
||||||
|
# Debian Packages Repository |
||||||
|
|
||||||
|
Publish [Debian](https://www.debian.org/distrib/packages) packages for your user or organization. |
||||||
|
|
||||||
|
**Table of Contents** |
||||||
|
|
||||||
|
{{< toc >}} |
||||||
|
|
||||||
|
## Requirements |
||||||
|
|
||||||
|
To work with the Debian registry, you need to use a HTTP client like `curl` to upload and a package manager like `apt` to consume packages. |
||||||
|
|
||||||
|
The following examples use `apt`. |
||||||
|
|
||||||
|
## Configuring the package registry |
||||||
|
|
||||||
|
To register the Debian registry add the url to the list of known apt sources: |
||||||
|
|
||||||
|
```shell |
||||||
|
echo "deb https://gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list |
||||||
|
``` |
||||||
|
|
||||||
|
| Placeholder | Description | |
||||||
|
| -------------- | ----------- | |
||||||
|
| `owner` | The owner of the package. | |
||||||
|
| `distribution` | The distribution to use. | |
||||||
|
| `component` | The component to use. | |
||||||
|
|
||||||
|
If the registry is private, provide credentials in the url. You can use a password or a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}): |
||||||
|
|
||||||
|
```shell |
||||||
|
echo "deb https://{username}:{your_password_or_token}@gitea.example.com/api/packages/{owner}/debian {distribution} {component}" | sudo tee -a /etc/apt/sources.list.d/gitea.list |
||||||
|
``` |
||||||
|
|
||||||
|
The Debian registry files are signed with a PGP key which must be known to apt: |
||||||
|
|
||||||
|
```shell |
||||||
|
sudo curl https://gitea.example.com/api/packages/{owner}/debian/repository.key -o /etc/apt/trusted.gpg.d/gitea-{owner}.asc |
||||||
|
``` |
||||||
|
|
||||||
|
Afterwards update the local package index: |
||||||
|
|
||||||
|
```shell |
||||||
|
apt update |
||||||
|
``` |
||||||
|
|
||||||
|
## Publish a package |
||||||
|
|
||||||
|
To publish a Debian package (`*.deb`), perform a HTTP `PUT` operation with the package content in the request body. |
||||||
|
|
||||||
|
``` |
||||||
|
PUT https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/upload |
||||||
|
``` |
||||||
|
|
||||||
|
| Parameter | Description | |
||||||
|
| -------------- | ----------- | |
||||||
|
| `owner` | The owner of the package. | |
||||||
|
| `distribution` | The distribution may match the release name of the OS, ex: `bionic`. | |
||||||
|
| `component` | The component can be used to group packages or just `main` or similar. | |
||||||
|
|
||||||
|
Example request using HTTP Basic authentication: |
||||||
|
|
||||||
|
```shell |
||||||
|
curl --user your_username:your_password_or_token \ |
||||||
|
--upload-file path/to/file.deb \ |
||||||
|
https://gitea.example.com/api/packages/testuser/debian/pool/bionic/main/upload |
||||||
|
``` |
||||||
|
|
||||||
|
If you are using 2FA or OAuth use a [personal access token]({{< relref "doc/development/api-usage.en-us.md#authentication" >}}) instead of the password. |
||||||
|
You cannot publish a file with the same name twice to a package. You must delete the existing package version first. |
||||||
|
|
||||||
|
The server reponds with the following HTTP Status codes. |
||||||
|
|
||||||
|
| HTTP Status Code | Meaning | |
||||||
|
| ----------------- | ------- | |
||||||
|
| `201 Created` | The package has been published. | |
||||||
|
| `400 Bad Request` | The package name, version, distribution, component or architecture are invalid. | |
||||||
|
| `409 Conflict` | A package file with the same combination of parameters exists already. | |
||||||
|
|
||||||
|
## Delete a package |
||||||
|
|
||||||
|
To delete a Debian package perform a HTTP `DELETE` operation. This will delete the package version too if there is no file left. |
||||||
|
|
||||||
|
``` |
||||||
|
DELETE https://gitea.example.com/api/packages/{owner}/debian/pool/{distribution}/{component}/{package_name}/{package_version}/{architecture} |
||||||
|
``` |
||||||
|
|
||||||
|
| Parameter | Description | |
||||||
|
| ----------------- | ----------- | |
||||||
|
| `owner` | The owner of the package. | |
||||||
|
| `package_name` | The package name. | |
||||||
|
| `package_version` | The package version. | |
||||||
|
| `distribution` | The package distribution. | |
||||||
|
| `component` | The package component. | |
||||||
|
| `architecture` | The package architecture. | |
||||||
|
|
||||||
|
Example request using HTTP Basic authentication: |
||||||
|
|
||||||
|
```shell |
||||||
|
curl --user your_username:your_token_or_password -X DELETE \ |
||||||
|
https://gitea.example.com/api/packages/testuser/debian/pools/bionic/main/test-package/1.0.0/amd64 |
||||||
|
``` |
||||||
|
|
||||||
|
The server reponds with the following HTTP Status codes. |
||||||
|
|
||||||
|
| HTTP Status Code | Meaning | |
||||||
|
| ----------------- | ------- | |
||||||
|
| `204 No Content` | Success | |
||||||
|
| `404 Not Found` | The package or file was not found. | |
||||||
|
|
||||||
|
## Install a package |
||||||
|
|
||||||
|
To install a package from the Debian registry, execute the following commands: |
||||||
|
|
||||||
|
```shell |
||||||
|
# use latest version |
||||||
|
apt install {package_name} |
||||||
|
# use specific version |
||||||
|
apt install {package_name}={package_version} |
||||||
|
``` |
||||||
@ -0,0 +1,23 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package v1_20 //nolint
|
||||||
|
|
||||||
|
import ( |
||||||
|
"xorm.io/xorm" |
||||||
|
) |
||||||
|
|
||||||
|
func AddIsInternalColumnToPackage(x *xorm.Engine) error { |
||||||
|
type Package struct { |
||||||
|
ID int64 `xorm:"pk autoincr"` |
||||||
|
OwnerID int64 `xorm:"UNIQUE(s) INDEX NOT NULL"` |
||||||
|
RepoID int64 `xorm:"INDEX"` |
||||||
|
Type string `xorm:"UNIQUE(s) INDEX NOT NULL"` |
||||||
|
Name string `xorm:"NOT NULL"` |
||||||
|
LowerName string `xorm:"UNIQUE(s) INDEX NOT NULL"` |
||||||
|
SemverCompatible bool `xorm:"NOT NULL DEFAULT false"` |
||||||
|
IsInternal bool `xorm:"NOT NULL DEFAULT false"` |
||||||
|
} |
||||||
|
|
||||||
|
return x.Sync(new(Package)) |
||||||
|
} |
||||||
@ -0,0 +1,131 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package debian |
||||||
|
|
||||||
|
import ( |
||||||
|
"context" |
||||||
|
"strconv" |
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db" |
||||||
|
"code.gitea.io/gitea/models/packages" |
||||||
|
debian_module "code.gitea.io/gitea/modules/packages/debian" |
||||||
|
|
||||||
|
"xorm.io/builder" |
||||||
|
) |
||||||
|
|
||||||
|
type PackageSearchOptions struct { |
||||||
|
OwnerID int64 |
||||||
|
Distribution string |
||||||
|
Component string |
||||||
|
Architecture string |
||||||
|
} |
||||||
|
|
||||||
|
// SearchLatestPackages gets the latest packages matching the search options
|
||||||
|
func SearchLatestPackages(ctx context.Context, opts *PackageSearchOptions) ([]*packages.PackageFileDescriptor, error) { |
||||||
|
var cond builder.Cond = builder.Eq{ |
||||||
|
"package_file.is_lead": true, |
||||||
|
"package.type": packages.TypeDebian, |
||||||
|
"package.owner_id": opts.OwnerID, |
||||||
|
"package.is_internal": false, |
||||||
|
"package_version.is_internal": false, |
||||||
|
} |
||||||
|
|
||||||
|
props := make(map[string]string) |
||||||
|
if opts.Distribution != "" { |
||||||
|
props[debian_module.PropertyDistribution] = opts.Distribution |
||||||
|
} |
||||||
|
if opts.Component != "" { |
||||||
|
props[debian_module.PropertyComponent] = opts.Component |
||||||
|
} |
||||||
|
if opts.Architecture != "" { |
||||||
|
props[debian_module.PropertyArchitecture] = opts.Architecture |
||||||
|
} |
||||||
|
|
||||||
|
if len(props) > 0 { |
||||||
|
var propsCond builder.Cond = builder.Eq{ |
||||||
|
"package_property.ref_type": packages.PropertyTypeFile, |
||||||
|
} |
||||||
|
propsCond = propsCond.And(builder.Expr("package_property.ref_id = package_file.id")) |
||||||
|
|
||||||
|
propsCondBlock := builder.NewCond() |
||||||
|
for name, value := range props { |
||||||
|
propsCondBlock = propsCondBlock.Or(builder.Eq{ |
||||||
|
"package_property.name": name, |
||||||
|
"package_property.value": value, |
||||||
|
}) |
||||||
|
} |
||||||
|
propsCond = propsCond.And(propsCondBlock) |
||||||
|
|
||||||
|
cond = cond.And(builder.Eq{ |
||||||
|
strconv.Itoa(len(props)): builder.Select("COUNT(*)").Where(propsCond).From("package_property"), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
cond = cond. |
||||||
|
And(builder.Expr("pv2.id IS NULL")) |
||||||
|
|
||||||
|
joinCond := builder. |
||||||
|
Expr("package_version.package_id = pv2.package_id AND (package_version.created_unix < pv2.created_unix OR (package_version.created_unix = pv2.created_unix AND package_version.id < pv2.id))"). |
||||||
|
And(builder.Eq{"pv2.is_internal": false}) |
||||||
|
|
||||||
|
pfs := make([]*packages.PackageFile, 0, 10) |
||||||
|
err := db.GetEngine(ctx). |
||||||
|
Table("package_file"). |
||||||
|
Select("package_file.*"). |
||||||
|
Join("INNER", "package_version", "package_version.id = package_file.version_id"). |
||||||
|
Join("LEFT", "package_version pv2", joinCond). |
||||||
|
Join("INNER", "package", "package.id = package_version.package_id"). |
||||||
|
Where(cond). |
||||||
|
Desc("package_version.created_unix"). |
||||||
|
Find(&pfs) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
return packages.GetPackageFileDescriptors(ctx, pfs) |
||||||
|
} |
||||||
|
|
||||||
|
// GetDistributions gets all available distributions
|
||||||
|
func GetDistributions(ctx context.Context, ownerID int64) ([]string, error) { |
||||||
|
return getDistinctPropertyValues(ctx, ownerID, "", debian_module.PropertyDistribution) |
||||||
|
} |
||||||
|
|
||||||
|
// GetComponents gets all available components for the given distribution
|
||||||
|
func GetComponents(ctx context.Context, ownerID int64, distribution string) ([]string, error) { |
||||||
|
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyComponent) |
||||||
|
} |
||||||
|
|
||||||
|
// GetArchitectures gets all available architectures for the given distribution
|
||||||
|
func GetArchitectures(ctx context.Context, ownerID int64, distribution string) ([]string, error) { |
||||||
|
return getDistinctPropertyValues(ctx, ownerID, distribution, debian_module.PropertyArchitecture) |
||||||
|
} |
||||||
|
|
||||||
|
func getDistinctPropertyValues(ctx context.Context, ownerID int64, distribution, propName string) ([]string, error) { |
||||||
|
var cond builder.Cond = builder.Eq{ |
||||||
|
"package_property.ref_type": packages.PropertyTypeFile, |
||||||
|
"package_property.name": propName, |
||||||
|
"package.type": packages.TypeDebian, |
||||||
|
"package.owner_id": ownerID, |
||||||
|
} |
||||||
|
if distribution != "" { |
||||||
|
innerCond := builder. |
||||||
|
Expr("pp.ref_id = package_property.ref_id"). |
||||||
|
And(builder.Eq{ |
||||||
|
"pp.ref_type": packages.PropertyTypeFile, |
||||||
|
"pp.name": debian_module.PropertyDistribution, |
||||||
|
"pp.value": distribution, |
||||||
|
}) |
||||||
|
cond = cond.And(builder.Exists(builder.Select("pp.ref_id").From("package_property pp").Where(innerCond))) |
||||||
|
} |
||||||
|
|
||||||
|
values := make([]string, 0, 5) |
||||||
|
return values, db.GetEngine(ctx). |
||||||
|
Table("package_property"). |
||||||
|
Distinct("package_property.value"). |
||||||
|
Join("INNER", "package_file", "package_file.id = package_property.ref_id"). |
||||||
|
Join("INNER", "package_version", "package_version.id = package_file.version_id"). |
||||||
|
Join("INNER", "package", "package.id = package_version.package_id"). |
||||||
|
Where(cond). |
||||||
|
Find(&values) |
||||||
|
} |
||||||
@ -0,0 +1,218 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package debian |
||||||
|
|
||||||
|
import ( |
||||||
|
"archive/tar" |
||||||
|
"bufio" |
||||||
|
"compress/gzip" |
||||||
|
"io" |
||||||
|
"net/mail" |
||||||
|
"regexp" |
||||||
|
"strings" |
||||||
|
|
||||||
|
"code.gitea.io/gitea/modules/util" |
||||||
|
"code.gitea.io/gitea/modules/validation" |
||||||
|
|
||||||
|
"github.com/blakesmith/ar" |
||||||
|
"github.com/klauspost/compress/zstd" |
||||||
|
"github.com/ulikunitz/xz" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
PropertyDistribution = "debian.distribution" |
||||||
|
PropertyComponent = "debian.component" |
||||||
|
PropertyArchitecture = "debian.architecture" |
||||||
|
PropertyControl = "debian.control" |
||||||
|
PropertyRepositoryIncludeInRelease = "debian.repository.include_in_release" |
||||||
|
|
||||||
|
SettingKeyPrivate = "debian.key.private" |
||||||
|
SettingKeyPublic = "debian.key.public" |
||||||
|
|
||||||
|
RepositoryPackage = "_debian" |
||||||
|
RepositoryVersion = "_repository" |
||||||
|
|
||||||
|
controlTar = "control.tar" |
||||||
|
) |
||||||
|
|
||||||
|
var ( |
||||||
|
ErrMissingControlFile = util.NewInvalidArgumentErrorf("control file is missing") |
||||||
|
ErrUnsupportedCompression = util.NewInvalidArgumentErrorf("unsupported compression algorithm") |
||||||
|
ErrInvalidName = util.NewInvalidArgumentErrorf("package name is invalid") |
||||||
|
ErrInvalidVersion = util.NewInvalidArgumentErrorf("package version is invalid") |
||||||
|
ErrInvalidArchitecture = util.NewInvalidArgumentErrorf("package architecture is invalid") |
||||||
|
|
||||||
|
// https://www.debian.org/doc/debian-policy/ch-controlfields.html#source
|
||||||
|
namePattern = regexp.MustCompile(`\A[a-z0-9][a-z0-9+-.]+\z`) |
||||||
|
// https://www.debian.org/doc/debian-policy/ch-controlfields.html#version
|
||||||
|
versionPattern = regexp.MustCompile(`\A(?:[0-9]:)?[a-zA-Z0-9.+~]+(?:-[a-zA-Z0-9.+-~]+)?\z`) |
||||||
|
) |
||||||
|
|
||||||
|
type Package struct { |
||||||
|
Name string |
||||||
|
Version string |
||||||
|
Architecture string |
||||||
|
Control string |
||||||
|
Metadata *Metadata |
||||||
|
} |
||||||
|
|
||||||
|
type Metadata struct { |
||||||
|
Maintainer string `json:"maintainer,omitempty"` |
||||||
|
ProjectURL string `json:"project_url,omitempty"` |
||||||
|
Description string `json:"description,omitempty"` |
||||||
|
Dependencies []string `json:"dependencies,omitempty"` |
||||||
|
} |
||||||
|
|
||||||
|
// ParsePackage parses the Debian package file
|
||||||
|
// https://manpages.debian.org/bullseye/dpkg-dev/deb.5.en.html
|
||||||
|
func ParsePackage(r io.Reader) (*Package, error) { |
||||||
|
arr := ar.NewReader(r) |
||||||
|
|
||||||
|
for { |
||||||
|
hd, err := arr.Next() |
||||||
|
if err == io.EOF { |
||||||
|
break |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
if strings.HasPrefix(hd.Name, controlTar) { |
||||||
|
var inner io.Reader |
||||||
|
switch hd.Name[len(controlTar):] { |
||||||
|
case "": |
||||||
|
inner = arr |
||||||
|
case ".gz": |
||||||
|
gzr, err := gzip.NewReader(arr) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
defer gzr.Close() |
||||||
|
|
||||||
|
inner = gzr |
||||||
|
case ".xz": |
||||||
|
xzr, err := xz.NewReader(arr) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
inner = xzr |
||||||
|
case ".zst": |
||||||
|
zr, err := zstd.NewReader(arr) |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
defer zr.Close() |
||||||
|
|
||||||
|
inner = zr |
||||||
|
default: |
||||||
|
return nil, ErrUnsupportedCompression |
||||||
|
} |
||||||
|
|
||||||
|
tr := tar.NewReader(inner) |
||||||
|
for { |
||||||
|
hd, err := tr.Next() |
||||||
|
if err == io.EOF { |
||||||
|
break |
||||||
|
} |
||||||
|
if err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
if hd.Typeflag != tar.TypeReg { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
if hd.FileInfo().Name() == "control" { |
||||||
|
return ParseControlFile(tr) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil, ErrMissingControlFile |
||||||
|
} |
||||||
|
|
||||||
|
// ParseControlFile parses a Debian control file to retrieve the metadata
|
||||||
|
func ParseControlFile(r io.Reader) (*Package, error) { |
||||||
|
p := &Package{ |
||||||
|
Metadata: &Metadata{}, |
||||||
|
} |
||||||
|
|
||||||
|
key := "" |
||||||
|
var depends strings.Builder |
||||||
|
var control strings.Builder |
||||||
|
|
||||||
|
s := bufio.NewScanner(io.TeeReader(r, &control)) |
||||||
|
for s.Scan() { |
||||||
|
line := s.Text() |
||||||
|
|
||||||
|
trimmed := strings.TrimSpace(line) |
||||||
|
if trimmed == "" { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
if line[0] == ' ' || line[0] == '\t' { |
||||||
|
switch key { |
||||||
|
case "Description": |
||||||
|
p.Metadata.Description += line |
||||||
|
case "Depends": |
||||||
|
depends.WriteString(trimmed) |
||||||
|
} |
||||||
|
} else { |
||||||
|
parts := strings.SplitN(trimmed, ":", 2) |
||||||
|
if len(parts) < 2 { |
||||||
|
continue |
||||||
|
} |
||||||
|
|
||||||
|
key = parts[0] |
||||||
|
value := strings.TrimSpace(parts[1]) |
||||||
|
switch key { |
||||||
|
case "Package": |
||||||
|
if !namePattern.MatchString(value) { |
||||||
|
return nil, ErrInvalidName |
||||||
|
} |
||||||
|
p.Name = value |
||||||
|
case "Version": |
||||||
|
if !versionPattern.MatchString(value) { |
||||||
|
return nil, ErrInvalidVersion |
||||||
|
} |
||||||
|
p.Version = value |
||||||
|
case "Architecture": |
||||||
|
if value == "" { |
||||||
|
return nil, ErrInvalidArchitecture |
||||||
|
} |
||||||
|
p.Architecture = value |
||||||
|
case "Maintainer": |
||||||
|
a, err := mail.ParseAddress(value) |
||||||
|
if err != nil || a.Name == "" { |
||||||
|
p.Metadata.Maintainer = value |
||||||
|
} else { |
||||||
|
p.Metadata.Maintainer = a.Name |
||||||
|
} |
||||||
|
case "Description": |
||||||
|
p.Metadata.Description = value |
||||||
|
case "Depends": |
||||||
|
depends.WriteString(value) |
||||||
|
case "Homepage": |
||||||
|
if validation.IsValidURL(value) { |
||||||
|
p.Metadata.ProjectURL = value |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
if err := s.Err(); err != nil { |
||||||
|
return nil, err |
||||||
|
} |
||||||
|
|
||||||
|
dependencies := strings.Split(depends.String(), ",") |
||||||
|
for i := range dependencies { |
||||||
|
dependencies[i] = strings.TrimSpace(dependencies[i]) |
||||||
|
} |
||||||
|
p.Metadata.Dependencies = dependencies |
||||||
|
|
||||||
|
p.Control = control.String() |
||||||
|
|
||||||
|
return p, nil |
||||||
|
} |
||||||
@ -0,0 +1,171 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package debian |
||||||
|
|
||||||
|
import ( |
||||||
|
"archive/tar" |
||||||
|
"bytes" |
||||||
|
"compress/gzip" |
||||||
|
"io" |
||||||
|
"testing" |
||||||
|
|
||||||
|
"github.com/blakesmith/ar" |
||||||
|
"github.com/klauspost/compress/zstd" |
||||||
|
"github.com/stretchr/testify/assert" |
||||||
|
"github.com/ulikunitz/xz" |
||||||
|
) |
||||||
|
|
||||||
|
const ( |
||||||
|
packageName = "gitea" |
||||||
|
packageVersion = "0:1.0.1-te~st" |
||||||
|
packageArchitecture = "amd64" |
||||||
|
packageAuthor = "KN4CK3R" |
||||||
|
description = "Description with multiple lines." |
||||||
|
projectURL = "https://gitea.io" |
||||||
|
) |
||||||
|
|
||||||
|
func TestParsePackage(t *testing.T) { |
||||||
|
createArchive := func(files map[string][]byte) io.Reader { |
||||||
|
var buf bytes.Buffer |
||||||
|
aw := ar.NewWriter(&buf) |
||||||
|
aw.WriteGlobalHeader() |
||||||
|
for filename, content := range files { |
||||||
|
hdr := &ar.Header{ |
||||||
|
Name: filename, |
||||||
|
Mode: 0o600, |
||||||
|
Size: int64(len(content)), |
||||||
|
} |
||||||
|
aw.WriteHeader(hdr) |
||||||
|
aw.Write(content) |
||||||
|
} |
||||||
|
return &buf |
||||||
|
} |
||||||
|
|
||||||
|
t.Run("MissingControlFile", func(t *testing.T) { |
||||||
|
data := createArchive(map[string][]byte{"dummy.txt": {}}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.Nil(t, p) |
||||||
|
assert.ErrorIs(t, err, ErrMissingControlFile) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("Compression", func(t *testing.T) { |
||||||
|
t.Run("Unsupported", func(t *testing.T) { |
||||||
|
data := createArchive(map[string][]byte{"control.tar.foo": {}}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.Nil(t, p) |
||||||
|
assert.ErrorIs(t, err, ErrUnsupportedCompression) |
||||||
|
}) |
||||||
|
|
||||||
|
var buf bytes.Buffer |
||||||
|
tw := tar.NewWriter(&buf) |
||||||
|
tw.WriteHeader(&tar.Header{ |
||||||
|
Name: "control", |
||||||
|
Mode: 0o600, |
||||||
|
Size: 50, |
||||||
|
}) |
||||||
|
tw.Write([]byte("Package: gitea\nVersion: 1.0.0\nArchitecture: amd64\n")) |
||||||
|
tw.Close() |
||||||
|
|
||||||
|
t.Run("None", func(t *testing.T) { |
||||||
|
data := createArchive(map[string][]byte{"control.tar": buf.Bytes()}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.NotNil(t, p) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Equal(t, "gitea", p.Name) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("gz", func(t *testing.T) { |
||||||
|
var zbuf bytes.Buffer |
||||||
|
zw := gzip.NewWriter(&zbuf) |
||||||
|
zw.Write(buf.Bytes()) |
||||||
|
zw.Close() |
||||||
|
|
||||||
|
data := createArchive(map[string][]byte{"control.tar.gz": zbuf.Bytes()}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.NotNil(t, p) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Equal(t, "gitea", p.Name) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("xz", func(t *testing.T) { |
||||||
|
var xbuf bytes.Buffer |
||||||
|
xw, _ := xz.NewWriter(&xbuf) |
||||||
|
xw.Write(buf.Bytes()) |
||||||
|
xw.Close() |
||||||
|
|
||||||
|
data := createArchive(map[string][]byte{"control.tar.xz": xbuf.Bytes()}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.NotNil(t, p) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Equal(t, "gitea", p.Name) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("zst", func(t *testing.T) { |
||||||
|
var zbuf bytes.Buffer |
||||||
|
zw, _ := zstd.NewWriter(&zbuf) |
||||||
|
zw.Write(buf.Bytes()) |
||||||
|
zw.Close() |
||||||
|
|
||||||
|
data := createArchive(map[string][]byte{"control.tar.zst": zbuf.Bytes()}) |
||||||
|
|
||||||
|
p, err := ParsePackage(data) |
||||||
|
assert.NotNil(t, p) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Equal(t, "gitea", p.Name) |
||||||
|
}) |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
func TestParseControlFile(t *testing.T) { |
||||||
|
buildContent := func(name, version, architecture string) *bytes.Buffer { |
||||||
|
var buf bytes.Buffer |
||||||
|
buf.WriteString("Package: " + name + "\nVersion: " + version + "\nArchitecture: " + architecture + "\nMaintainer: " + packageAuthor + " <kn4ck3r@gitea.io>\nHomepage: " + projectURL + "\nDepends: a,\n b\nDescription: Description\n with multiple\n lines.") |
||||||
|
return &buf |
||||||
|
} |
||||||
|
|
||||||
|
t.Run("InvalidName", func(t *testing.T) { |
||||||
|
for _, name := range []string{"", "-cd"} { |
||||||
|
p, err := ParseControlFile(buildContent(name, packageVersion, packageArchitecture)) |
||||||
|
assert.Nil(t, p) |
||||||
|
assert.ErrorIs(t, err, ErrInvalidName) |
||||||
|
} |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("InvalidVersion", func(t *testing.T) { |
||||||
|
for _, version := range []string{"", "1-", ":1.0", "1_0"} { |
||||||
|
p, err := ParseControlFile(buildContent(packageName, version, packageArchitecture)) |
||||||
|
assert.Nil(t, p) |
||||||
|
assert.ErrorIs(t, err, ErrInvalidVersion) |
||||||
|
} |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("InvalidArchitecture", func(t *testing.T) { |
||||||
|
p, err := ParseControlFile(buildContent(packageName, packageVersion, "")) |
||||||
|
assert.Nil(t, p) |
||||||
|
assert.ErrorIs(t, err, ErrInvalidArchitecture) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("Valid", func(t *testing.T) { |
||||||
|
content := buildContent(packageName, packageVersion, packageArchitecture) |
||||||
|
full := content.String() |
||||||
|
|
||||||
|
p, err := ParseControlFile(content) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.NotNil(t, p) |
||||||
|
|
||||||
|
assert.Equal(t, packageName, p.Name) |
||||||
|
assert.Equal(t, packageVersion, p.Version) |
||||||
|
assert.Equal(t, packageArchitecture, p.Architecture) |
||||||
|
assert.Equal(t, description, p.Metadata.Description) |
||||||
|
assert.Equal(t, projectURL, p.Metadata.ProjectURL) |
||||||
|
assert.Equal(t, packageAuthor, p.Metadata.Maintainer) |
||||||
|
assert.Equal(t, []string{"a", "b"}, p.Metadata.Dependencies) |
||||||
|
assert.Equal(t, full, p.Control) |
||||||
|
}) |
||||||
|
} |
||||||
|
After Width: | Height: | Size: 4.2 KiB |
@ -0,0 +1,317 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package debian |
||||||
|
|
||||||
|
import ( |
||||||
|
stdctx "context" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net/http" |
||||||
|
"strings" |
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db" |
||||||
|
packages_model "code.gitea.io/gitea/models/packages" |
||||||
|
"code.gitea.io/gitea/modules/context" |
||||||
|
"code.gitea.io/gitea/modules/notification" |
||||||
|
packages_module "code.gitea.io/gitea/modules/packages" |
||||||
|
debian_module "code.gitea.io/gitea/modules/packages/debian" |
||||||
|
"code.gitea.io/gitea/modules/util" |
||||||
|
"code.gitea.io/gitea/routers/api/packages/helper" |
||||||
|
packages_service "code.gitea.io/gitea/services/packages" |
||||||
|
debian_service "code.gitea.io/gitea/services/packages/debian" |
||||||
|
) |
||||||
|
|
||||||
|
func apiError(ctx *context.Context, status int, obj interface{}) { |
||||||
|
helper.LogAndProcessError(ctx, status, obj, func(message string) { |
||||||
|
ctx.PlainText(status, message) |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
func GetRepositoryKey(ctx *context.Context) { |
||||||
|
_, pub, err := debian_service.GetOrCreateKeyPair(ctx.Package.Owner.ID) |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
ctx.ServeContent(strings.NewReader(pub), &context.ServeHeaderOptions{ |
||||||
|
ContentType: "application/pgp-keys", |
||||||
|
Filename: "repository.key", |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files
|
||||||
|
// https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices
|
||||||
|
func GetRepositoryFile(ctx *context.Context) { |
||||||
|
pv, err := debian_service.GetOrCreateRepositoryVersion(ctx.Package.Owner.ID) |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
key := ctx.Params("distribution") |
||||||
|
|
||||||
|
component := ctx.Params("component") |
||||||
|
architecture := strings.TrimPrefix(ctx.Params("architecture"), "binary-") |
||||||
|
if component != "" && architecture != "" { |
||||||
|
key += "|" + component + "|" + architecture |
||||||
|
} |
||||||
|
|
||||||
|
s, pf, err := packages_service.GetFileStreamByPackageVersion( |
||||||
|
ctx, |
||||||
|
pv, |
||||||
|
&packages_service.PackageFileInfo{ |
||||||
|
Filename: ctx.Params("filename"), |
||||||
|
CompositeKey: key, |
||||||
|
}, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
if err == packages_model.ErrPackageNotExist || err == packages_model.ErrPackageFileNotExist { |
||||||
|
apiError(ctx, http.StatusNotFound, err) |
||||||
|
} else { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
defer s.Close() |
||||||
|
|
||||||
|
ctx.ServeContent(s, &context.ServeHeaderOptions{ |
||||||
|
Filename: pf.Name, |
||||||
|
LastModified: pf.CreatedUnix.AsLocalTime(), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// https://wiki.debian.org/DebianRepository/Format#indices_acquisition_via_hashsums_.28by-hash.29
|
||||||
|
func GetRepositoryFileByHash(ctx *context.Context) { |
||||||
|
pv, err := debian_service.GetOrCreateRepositoryVersion(ctx.Package.Owner.ID) |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
algorithm := strings.ToLower(ctx.Params("algorithm")) |
||||||
|
if algorithm == "md5sum" { |
||||||
|
algorithm = "md5" |
||||||
|
} |
||||||
|
|
||||||
|
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{ |
||||||
|
VersionID: pv.ID, |
||||||
|
Hash: strings.ToLower(ctx.Params("hash")), |
||||||
|
HashAlgorithm: algorithm, |
||||||
|
}) |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
if len(pfs) != 1 { |
||||||
|
apiError(ctx, http.StatusNotFound, nil) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
s, pf, err := packages_service.GetPackageFileStream(ctx, pfs[0]) |
||||||
|
if err != nil { |
||||||
|
if errors.Is(err, util.ErrNotExist) { |
||||||
|
apiError(ctx, http.StatusNotFound, err) |
||||||
|
} else { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
defer s.Close() |
||||||
|
|
||||||
|
ctx.ServeContent(s, &context.ServeHeaderOptions{ |
||||||
|
Filename: pf.Name, |
||||||
|
LastModified: pf.CreatedUnix.AsLocalTime(), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
func UploadPackageFile(ctx *context.Context) { |
||||||
|
distribution := strings.TrimSpace(ctx.Params("distribution")) |
||||||
|
component := strings.TrimSpace(ctx.Params("component")) |
||||||
|
if distribution == "" || component == "" { |
||||||
|
apiError(ctx, http.StatusBadRequest, "invalid distribution or component") |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
upload, close, err := ctx.UploadStream() |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
if close { |
||||||
|
defer upload.Close() |
||||||
|
} |
||||||
|
|
||||||
|
buf, err := packages_module.CreateHashedBufferFromReader(upload) |
||||||
|
if err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
defer buf.Close() |
||||||
|
|
||||||
|
pck, err := debian_module.ParsePackage(buf) |
||||||
|
if err != nil { |
||||||
|
if errors.Is(err, util.ErrInvalidArgument) { |
||||||
|
apiError(ctx, http.StatusBadRequest, err) |
||||||
|
} else { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
if _, err := buf.Seek(0, io.SeekStart); err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
_, _, err = packages_service.CreatePackageOrAddFileToExisting( |
||||||
|
&packages_service.PackageCreationInfo{ |
||||||
|
PackageInfo: packages_service.PackageInfo{ |
||||||
|
Owner: ctx.Package.Owner, |
||||||
|
PackageType: packages_model.TypeDebian, |
||||||
|
Name: pck.Name, |
||||||
|
Version: pck.Version, |
||||||
|
}, |
||||||
|
Creator: ctx.Doer, |
||||||
|
Metadata: pck.Metadata, |
||||||
|
}, |
||||||
|
&packages_service.PackageFileCreationInfo{ |
||||||
|
PackageFileInfo: packages_service.PackageFileInfo{ |
||||||
|
Filename: fmt.Sprintf("%s_%s_%s.deb", pck.Name, pck.Version, pck.Architecture), |
||||||
|
CompositeKey: fmt.Sprintf("%s|%s", distribution, component), |
||||||
|
}, |
||||||
|
Creator: ctx.Doer, |
||||||
|
Data: buf, |
||||||
|
IsLead: true, |
||||||
|
Properties: map[string]string{ |
||||||
|
debian_module.PropertyDistribution: distribution, |
||||||
|
debian_module.PropertyComponent: component, |
||||||
|
debian_module.PropertyArchitecture: pck.Architecture, |
||||||
|
debian_module.PropertyControl: pck.Control, |
||||||
|
}, |
||||||
|
}, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
switch err { |
||||||
|
case packages_model.ErrDuplicatePackageVersion: |
||||||
|
apiError(ctx, http.StatusBadRequest, err) |
||||||
|
case packages_service.ErrQuotaTotalCount, packages_service.ErrQuotaTypeSize, packages_service.ErrQuotaTotalSize: |
||||||
|
apiError(ctx, http.StatusForbidden, err) |
||||||
|
default: |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
if err := debian_service.BuildSpecificRepositoryFiles(ctx, ctx.Package.Owner.ID, distribution, component, pck.Architecture); err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
ctx.Status(http.StatusCreated) |
||||||
|
} |
||||||
|
|
||||||
|
func DownloadPackageFile(ctx *context.Context) { |
||||||
|
name := ctx.Params("name") |
||||||
|
version := ctx.Params("version") |
||||||
|
|
||||||
|
s, pf, err := packages_service.GetFileStreamByPackageNameAndVersion( |
||||||
|
ctx, |
||||||
|
&packages_service.PackageInfo{ |
||||||
|
Owner: ctx.Package.Owner, |
||||||
|
PackageType: packages_model.TypeDebian, |
||||||
|
Name: name, |
||||||
|
Version: version, |
||||||
|
}, |
||||||
|
&packages_service.PackageFileInfo{ |
||||||
|
Filename: fmt.Sprintf("%s_%s_%s.deb", name, version, ctx.Params("architecture")), |
||||||
|
CompositeKey: fmt.Sprintf("%s|%s", ctx.Params("distribution"), ctx.Params("component")), |
||||||
|
}, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
if errors.Is(err, util.ErrNotExist) { |
||||||
|
apiError(ctx, http.StatusNotFound, err) |
||||||
|
} else { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
defer s.Close() |
||||||
|
|
||||||
|
ctx.ServeContent(s, &context.ServeHeaderOptions{ |
||||||
|
ContentType: "application/vnd.debian.binary-package", |
||||||
|
Filename: pf.Name, |
||||||
|
LastModified: pf.CreatedUnix.AsLocalTime(), |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
func DeletePackageFile(ctx *context.Context) { |
||||||
|
distribution := ctx.Params("distribution") |
||||||
|
component := ctx.Params("component") |
||||||
|
name := ctx.Params("name") |
||||||
|
version := ctx.Params("version") |
||||||
|
architecture := ctx.Params("architecture") |
||||||
|
|
||||||
|
owner := ctx.Package.Owner |
||||||
|
|
||||||
|
var pd *packages_model.PackageDescriptor |
||||||
|
|
||||||
|
err := db.WithTx(ctx, func(ctx stdctx.Context) error { |
||||||
|
pv, err := packages_model.GetVersionByNameAndVersion(ctx, owner.ID, packages_model.TypeDebian, name, version) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
pf, err := packages_model.GetFileForVersionByName( |
||||||
|
ctx, |
||||||
|
pv.ID, |
||||||
|
fmt.Sprintf("%s_%s_%s.deb", name, version, architecture), |
||||||
|
fmt.Sprintf("%s|%s", distribution, component), |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if err := packages_service.DeletePackageFile(ctx, pf); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
has, err := packages_model.HasVersionFileReferences(ctx, pv.ID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if !has { |
||||||
|
pd, err = packages_model.GetPackageDescriptor(ctx, pv) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if err := packages_service.DeletePackageVersionAndReferences(ctx, pv); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
}) |
||||||
|
if err != nil { |
||||||
|
if errors.Is(err, util.ErrNotExist) { |
||||||
|
apiError(ctx, http.StatusNotFound, err) |
||||||
|
} else { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
} |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
if pd != nil { |
||||||
|
notification.NotifyPackageDelete(ctx, ctx.Doer, pd) |
||||||
|
} |
||||||
|
|
||||||
|
if err := debian_service.BuildSpecificRepositoryFiles(ctx, ctx.Package.Owner.ID, distribution, component, architecture); err != nil { |
||||||
|
apiError(ctx, http.StatusInternalServerError, err) |
||||||
|
return |
||||||
|
} |
||||||
|
|
||||||
|
ctx.Status(http.StatusNoContent) |
||||||
|
} |
||||||
@ -0,0 +1,443 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package debian |
||||||
|
|
||||||
|
import ( |
||||||
|
"bytes" |
||||||
|
"compress/gzip" |
||||||
|
"context" |
||||||
|
"errors" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"sort" |
||||||
|
"strings" |
||||||
|
"time" |
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db" |
||||||
|
packages_model "code.gitea.io/gitea/models/packages" |
||||||
|
debian_model "code.gitea.io/gitea/models/packages/debian" |
||||||
|
user_model "code.gitea.io/gitea/models/user" |
||||||
|
"code.gitea.io/gitea/modules/log" |
||||||
|
packages_module "code.gitea.io/gitea/modules/packages" |
||||||
|
debian_module "code.gitea.io/gitea/modules/packages/debian" |
||||||
|
"code.gitea.io/gitea/modules/setting" |
||||||
|
"code.gitea.io/gitea/modules/util" |
||||||
|
packages_service "code.gitea.io/gitea/services/packages" |
||||||
|
|
||||||
|
"github.com/keybase/go-crypto/openpgp" |
||||||
|
"github.com/keybase/go-crypto/openpgp/armor" |
||||||
|
"github.com/keybase/go-crypto/openpgp/clearsign" |
||||||
|
"github.com/keybase/go-crypto/openpgp/packet" |
||||||
|
"github.com/ulikunitz/xz" |
||||||
|
) |
||||||
|
|
||||||
|
// GetOrCreateRepositoryVersion gets or creates the internal repository package
|
||||||
|
// The Debian registry needs multiple index files which are stored in this package.
|
||||||
|
func GetOrCreateRepositoryVersion(ownerID int64) (*packages_model.PackageVersion, error) { |
||||||
|
var repositoryVersion *packages_model.PackageVersion |
||||||
|
|
||||||
|
return repositoryVersion, db.WithTx(db.DefaultContext, func(ctx context.Context) error { |
||||||
|
p := &packages_model.Package{ |
||||||
|
OwnerID: ownerID, |
||||||
|
Type: packages_model.TypeDebian, |
||||||
|
Name: debian_module.RepositoryPackage, |
||||||
|
LowerName: debian_module.RepositoryPackage, |
||||||
|
IsInternal: true, |
||||||
|
} |
||||||
|
var err error |
||||||
|
if p, err = packages_model.TryInsertPackage(ctx, p); err != nil { |
||||||
|
if err != packages_model.ErrDuplicatePackage { |
||||||
|
log.Error("Error inserting package: %v", err) |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
pv := &packages_model.PackageVersion{ |
||||||
|
PackageID: p.ID, |
||||||
|
CreatorID: ownerID, |
||||||
|
Version: debian_module.RepositoryVersion, |
||||||
|
LowerVersion: debian_module.RepositoryVersion, |
||||||
|
IsInternal: true, |
||||||
|
MetadataJSON: "null", |
||||||
|
} |
||||||
|
if pv, err = packages_model.GetOrInsertVersion(ctx, pv); err != nil { |
||||||
|
if err != packages_model.ErrDuplicatePackageVersion { |
||||||
|
log.Error("Error inserting package version: %v", err) |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
repositoryVersion = pv |
||||||
|
|
||||||
|
return nil |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
// GetOrCreateKeyPair gets or creates the PGP keys used to sign repository files
|
||||||
|
func GetOrCreateKeyPair(ownerID int64) (string, string, error) { |
||||||
|
priv, err := user_model.GetSetting(ownerID, debian_module.SettingKeyPrivate) |
||||||
|
if err != nil && !errors.Is(err, util.ErrNotExist) { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
|
||||||
|
pub, err := user_model.GetSetting(ownerID, debian_module.SettingKeyPublic) |
||||||
|
if err != nil && !errors.Is(err, util.ErrNotExist) { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
|
||||||
|
if priv == "" || pub == "" { |
||||||
|
priv, pub, err = generateKeypair() |
||||||
|
if err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
|
||||||
|
if err := user_model.SetUserSetting(ownerID, debian_module.SettingKeyPrivate, priv); err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
|
||||||
|
if err := user_model.SetUserSetting(ownerID, debian_module.SettingKeyPublic, pub); err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return priv, pub, nil |
||||||
|
} |
||||||
|
|
||||||
|
func generateKeypair() (string, string, error) { |
||||||
|
e, err := openpgp.NewEntity(setting.AppName, "Debian Registry", "", nil) |
||||||
|
if err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
|
||||||
|
var priv strings.Builder |
||||||
|
var pub strings.Builder |
||||||
|
|
||||||
|
w, err := armor.Encode(&priv, openpgp.PrivateKeyType, nil) |
||||||
|
if err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
if err := e.SerializePrivate(w, nil); err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
w.Close() |
||||||
|
|
||||||
|
w, err = armor.Encode(&pub, openpgp.PublicKeyType, nil) |
||||||
|
if err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
if err := e.Serialize(w); err != nil { |
||||||
|
return "", "", err |
||||||
|
} |
||||||
|
w.Close() |
||||||
|
|
||||||
|
return priv.String(), pub.String(), nil |
||||||
|
} |
||||||
|
|
||||||
|
// BuildAllRepositoryFiles (re)builds all repository files for every available distributions, components and architectures
|
||||||
|
func BuildAllRepositoryFiles(ctx context.Context, ownerID int64) error { |
||||||
|
pv, err := GetOrCreateRepositoryVersion(ownerID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// 1. Delete all existing repository files
|
||||||
|
pfs, err := packages_model.GetFilesByVersionID(ctx, pv.ID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
for _, pf := range pfs { |
||||||
|
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
// 2. (Re)Build repository files for existing packages
|
||||||
|
distributions, err := debian_model.GetDistributions(ctx, ownerID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
for _, distribution := range distributions { |
||||||
|
components, err := debian_model.GetComponents(ctx, ownerID, distribution) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
architectures, err := debian_model.GetArchitectures(ctx, ownerID, distribution) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
for _, component := range components { |
||||||
|
for _, architecture := range architectures { |
||||||
|
if err := buildRepositoryFiles(ctx, ownerID, pv, distribution, component, architecture); err != nil { |
||||||
|
return fmt.Errorf("failed to build repository files [%s/%s/%s]: %w", distribution, component, architecture, err) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// BuildSpecificRepositoryFiles builds index files for the repository
|
||||||
|
func BuildSpecificRepositoryFiles(ctx context.Context, ownerID int64, distribution, component, architecture string) error { |
||||||
|
pv, err := GetOrCreateRepositoryVersion(ownerID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
return buildRepositoryFiles(ctx, ownerID, pv, distribution, component, architecture) |
||||||
|
} |
||||||
|
|
||||||
|
func buildRepositoryFiles(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution, component, architecture string) error { |
||||||
|
if err := buildPackagesIndices(ctx, ownerID, repoVersion, distribution, component, architecture); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
return buildReleaseFiles(ctx, ownerID, repoVersion, distribution) |
||||||
|
} |
||||||
|
|
||||||
|
// https://wiki.debian.org/DebianRepository/Format#A.22Packages.22_Indices
|
||||||
|
func buildPackagesIndices(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution, component, architecture string) error { |
||||||
|
pfds, err := debian_model.SearchLatestPackages(ctx, &debian_model.PackageSearchOptions{ |
||||||
|
OwnerID: ownerID, |
||||||
|
Distribution: distribution, |
||||||
|
Component: component, |
||||||
|
Architecture: architecture, |
||||||
|
}) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Delete the package indices if there are no packages
|
||||||
|
if len(pfds) == 0 { |
||||||
|
key := fmt.Sprintf("%s|%s|%s", distribution, component, architecture) |
||||||
|
for _, filename := range []string{"Packages", "Packages.gz", "Packages.xz"} { |
||||||
|
pf, err := packages_model.GetFileForVersionByName(ctx, repoVersion.ID, filename, key) |
||||||
|
if err != nil && !errors.Is(err, util.ErrNotExist) { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
packagesContent, _ := packages_module.NewHashedBuffer() |
||||||
|
|
||||||
|
packagesGzipContent, _ := packages_module.NewHashedBuffer() |
||||||
|
gzw := gzip.NewWriter(packagesGzipContent) |
||||||
|
|
||||||
|
packagesXzContent, _ := packages_module.NewHashedBuffer() |
||||||
|
xzw, _ := xz.NewWriter(packagesXzContent) |
||||||
|
|
||||||
|
w := io.MultiWriter(packagesContent, gzw, xzw) |
||||||
|
|
||||||
|
addSeparator := false |
||||||
|
for _, pfd := range pfds { |
||||||
|
if addSeparator { |
||||||
|
fmt.Fprintln(w) |
||||||
|
} |
||||||
|
addSeparator = true |
||||||
|
|
||||||
|
fmt.Fprint(w, pfd.Properties.GetByName(debian_module.PropertyControl)) |
||||||
|
|
||||||
|
fmt.Fprintf(w, "Filename: pool/%s/%s/%s\n", distribution, component, pfd.File.Name) |
||||||
|
fmt.Fprintf(w, "Size: %d\n", pfd.Blob.Size) |
||||||
|
fmt.Fprintf(w, "MD5sum: %s\n", pfd.Blob.HashMD5) |
||||||
|
fmt.Fprintf(w, "SHA1: %s\n", pfd.Blob.HashSHA1) |
||||||
|
fmt.Fprintf(w, "SHA256: %s\n", pfd.Blob.HashSHA256) |
||||||
|
fmt.Fprintf(w, "SHA512: %s\n", pfd.Blob.HashSHA512) |
||||||
|
} |
||||||
|
|
||||||
|
gzw.Close() |
||||||
|
xzw.Close() |
||||||
|
|
||||||
|
for _, file := range []struct { |
||||||
|
Name string |
||||||
|
Data packages_module.HashedSizeReader |
||||||
|
}{ |
||||||
|
{"Packages", packagesContent}, |
||||||
|
{"Packages.gz", packagesGzipContent}, |
||||||
|
{"Packages.xz", packagesXzContent}, |
||||||
|
} { |
||||||
|
_, err = packages_service.AddFileToPackageVersionInternal( |
||||||
|
repoVersion, |
||||||
|
&packages_service.PackageFileCreationInfo{ |
||||||
|
PackageFileInfo: packages_service.PackageFileInfo{ |
||||||
|
Filename: file.Name, |
||||||
|
CompositeKey: fmt.Sprintf("%s|%s|%s", distribution, component, architecture), |
||||||
|
}, |
||||||
|
Creator: user_model.NewGhostUser(), |
||||||
|
Data: file.Data, |
||||||
|
IsLead: false, |
||||||
|
OverwriteExisting: true, |
||||||
|
Properties: map[string]string{ |
||||||
|
debian_module.PropertyRepositoryIncludeInRelease: "", |
||||||
|
debian_module.PropertyDistribution: distribution, |
||||||
|
debian_module.PropertyComponent: component, |
||||||
|
debian_module.PropertyArchitecture: architecture, |
||||||
|
}, |
||||||
|
}, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
// https://wiki.debian.org/DebianRepository/Format#A.22Release.22_files
|
||||||
|
func buildReleaseFiles(ctx context.Context, ownerID int64, repoVersion *packages_model.PackageVersion, distribution string) error { |
||||||
|
pfs, _, err := packages_model.SearchFiles(ctx, &packages_model.PackageFileSearchOptions{ |
||||||
|
VersionID: repoVersion.ID, |
||||||
|
Properties: map[string]string{ |
||||||
|
debian_module.PropertyRepositoryIncludeInRelease: "", |
||||||
|
debian_module.PropertyDistribution: distribution, |
||||||
|
}, |
||||||
|
}) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
// Delete the release files if there are no packages
|
||||||
|
if len(pfs) == 0 { |
||||||
|
for _, filename := range []string{"Release", "Release.gpg", "InRelease"} { |
||||||
|
pf, err := packages_model.GetFileForVersionByName(ctx, repoVersion.ID, filename, distribution) |
||||||
|
if err != nil && !errors.Is(err, util.ErrNotExist) { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
if err := packages_model.DeleteAllProperties(ctx, packages_model.PropertyTypeFile, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
if err := packages_model.DeleteFileByID(ctx, pf.ID); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
|
|
||||||
|
components, err := debian_model.GetComponents(ctx, ownerID, distribution) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
sort.Strings(components) |
||||||
|
|
||||||
|
architectures, err := debian_model.GetArchitectures(ctx, ownerID, distribution) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
sort.Strings(architectures) |
||||||
|
|
||||||
|
priv, _, err := GetOrCreateKeyPair(ownerID) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
block, err := armor.Decode(strings.NewReader(priv)) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
e, err := openpgp.ReadEntity(packet.NewReader(block.Body)) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
inReleaseContent, _ := packages_module.NewHashedBuffer() |
||||||
|
sw, err := clearsign.Encode(inReleaseContent, e.PrivateKey, nil) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
var buf bytes.Buffer |
||||||
|
|
||||||
|
w := io.MultiWriter(sw, &buf) |
||||||
|
|
||||||
|
fmt.Fprintf(w, "Origin: %s\n", setting.AppName) |
||||||
|
fmt.Fprintf(w, "Label: %s\n", setting.AppName) |
||||||
|
fmt.Fprintf(w, "Suite: %s\n", distribution) |
||||||
|
fmt.Fprintf(w, "Codename: %s\n", distribution) |
||||||
|
fmt.Fprintf(w, "Components: %s\n", strings.Join(components, " ")) |
||||||
|
fmt.Fprintf(w, "Architectures: %s\n", strings.Join(architectures, " ")) |
||||||
|
fmt.Fprintf(w, "Date: %s\n", time.Now().UTC().Format(time.RFC1123)) |
||||||
|
fmt.Fprint(w, "Acquire-By-Hash: yes") |
||||||
|
|
||||||
|
pfds, err := packages_model.GetPackageFileDescriptors(ctx, pfs) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
var md5, sha1, sha256, sha512 strings.Builder |
||||||
|
for _, pfd := range pfds { |
||||||
|
path := fmt.Sprintf("%s/binary-%s/%s", pfd.Properties.GetByName(debian_module.PropertyComponent), pfd.Properties.GetByName(debian_module.PropertyArchitecture), pfd.File.Name) |
||||||
|
fmt.Fprintf(&md5, " %s %d %s\n", pfd.Blob.HashMD5, pfd.Blob.Size, path) |
||||||
|
fmt.Fprintf(&sha1, " %s %d %s\n", pfd.Blob.HashSHA1, pfd.Blob.Size, path) |
||||||
|
fmt.Fprintf(&sha256, " %s %d %s\n", pfd.Blob.HashSHA256, pfd.Blob.Size, path) |
||||||
|
fmt.Fprintf(&sha512, " %s %d %s\n", pfd.Blob.HashSHA512, pfd.Blob.Size, path) |
||||||
|
} |
||||||
|
|
||||||
|
fmt.Fprintln(w, "MD5Sum:") |
||||||
|
fmt.Fprint(w, md5.String()) |
||||||
|
fmt.Fprintln(w, "SHA1:") |
||||||
|
fmt.Fprint(w, sha1.String()) |
||||||
|
fmt.Fprintln(w, "SHA256:") |
||||||
|
fmt.Fprint(w, sha256.String()) |
||||||
|
fmt.Fprintln(w, "SHA512:") |
||||||
|
fmt.Fprint(w, sha512.String()) |
||||||
|
|
||||||
|
sw.Close() |
||||||
|
|
||||||
|
releaseGpgContent, _ := packages_module.NewHashedBuffer() |
||||||
|
if err := openpgp.ArmoredDetachSign(releaseGpgContent, e, bytes.NewReader(buf.Bytes()), nil); err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
|
||||||
|
releaseContent, _ := packages_module.CreateHashedBufferFromReader(&buf) |
||||||
|
|
||||||
|
for _, file := range []struct { |
||||||
|
Name string |
||||||
|
Data packages_module.HashedSizeReader |
||||||
|
}{ |
||||||
|
{"Release", releaseContent}, |
||||||
|
{"Release.gpg", releaseGpgContent}, |
||||||
|
{"InRelease", inReleaseContent}, |
||||||
|
} { |
||||||
|
_, err = packages_service.AddFileToPackageVersionInternal( |
||||||
|
repoVersion, |
||||||
|
&packages_service.PackageFileCreationInfo{ |
||||||
|
PackageFileInfo: packages_service.PackageFileInfo{ |
||||||
|
Filename: file.Name, |
||||||
|
CompositeKey: distribution, |
||||||
|
}, |
||||||
|
Creator: user_model.NewGhostUser(), |
||||||
|
Data: file.Data, |
||||||
|
IsLead: false, |
||||||
|
OverwriteExisting: true, |
||||||
|
Properties: map[string]string{ |
||||||
|
debian_module.PropertyDistribution: distribution, |
||||||
|
}, |
||||||
|
}, |
||||||
|
) |
||||||
|
if err != nil { |
||||||
|
return err |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
return nil |
||||||
|
} |
||||||
@ -0,0 +1,65 @@ |
|||||||
|
{{if eq .PackageDescriptor.Package.Type "debian"}} |
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.installation"}}</h4> |
||||||
|
<div class="ui attached segment"> |
||||||
|
<div class="ui form"> |
||||||
|
<div class="field"> |
||||||
|
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.debian.registry"}}</label> |
||||||
|
<div class="markup"><pre class="code-block"><code>sudo curl <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/debian/repository.key"></gitea-origin-url> -o /etc/apt/trusted.gpg.d/gitea-{{$.PackageDescriptor.Owner.Name}}.asc |
||||||
|
echo "deb <gitea-origin-url data-url="{{AppSubUrl}}/api/packages/{{$.PackageDescriptor.Owner.Name}}/debian"></gitea-origin-url> $distribution $component" | sudo tee -a /etc/apt/sources.list.d/gitea.list |
||||||
|
sudo apt update</code></pre></div> |
||||||
|
<p>{{.locale.Tr "packages.debian.registry.info" | Safe}}</p> |
||||||
|
</div> |
||||||
|
<div class="field"> |
||||||
|
<label>{{svg "octicon-terminal"}} {{.locale.Tr "packages.debian.install"}}</label> |
||||||
|
<div class="markup"> |
||||||
|
<pre class="code-block"><code>sudo apt install {{$.PackageDescriptor.Package.Name}}={{$.PackageDescriptor.Version.Version}}</code></pre> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
<div class="field"> |
||||||
|
<label>{{.locale.Tr "packages.debian.documentation" "https://docs.gitea.io/en-us/packages/debian/" | Safe}}</label> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
</div> |
||||||
|
|
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.debian.repository"}}</h4> |
||||||
|
<div class="ui attached segment"> |
||||||
|
<table class="ui single line very basic table"> |
||||||
|
<tbody> |
||||||
|
<tr> |
||||||
|
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.distributions"}}</h5></td> |
||||||
|
<td>{{StringUtils.Join .Distributions ", "}}</td> |
||||||
|
</tr> |
||||||
|
<tr> |
||||||
|
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.components"}}</h5></td> |
||||||
|
<td>{{StringUtils.Join .Components ", "}}</td> |
||||||
|
</tr> |
||||||
|
<tr> |
||||||
|
<td class="collapsing"><h5>{{.locale.Tr "packages.debian.repository.architectures"}}</h5></td> |
||||||
|
<td>{{StringUtils.Join .Architectures ", "}}</td> |
||||||
|
</tr> |
||||||
|
</tbody> |
||||||
|
</table> |
||||||
|
</div> |
||||||
|
|
||||||
|
{{if .PackageDescriptor.Metadata.Description}} |
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.about"}}</h4> |
||||||
|
<div class="ui attached segment"> |
||||||
|
{{.PackageDescriptor.Metadata.Description}} |
||||||
|
</div> |
||||||
|
{{end}} |
||||||
|
|
||||||
|
{{if .PackageDescriptor.Metadata.Dependencies}} |
||||||
|
<h4 class="ui top attached header">{{.locale.Tr "packages.dependencies"}}</h4> |
||||||
|
<div class="ui attached segment"> |
||||||
|
<table class="ui single line very basic table"> |
||||||
|
<tbody> |
||||||
|
{{range .PackageDescriptor.Metadata.Dependencies}} |
||||||
|
<tr> |
||||||
|
<td>{{.}}</td> |
||||||
|
</tr> |
||||||
|
{{end}} |
||||||
|
</tbody> |
||||||
|
</table> |
||||||
|
</div> |
||||||
|
{{end}} |
||||||
|
{{end}} |
||||||
@ -0,0 +1,4 @@ |
|||||||
|
{{if eq .PackageDescriptor.Package.Type "debian"}} |
||||||
|
{{if .PackageDescriptor.Metadata.Maintainer}}<div class="item" title="{{.locale.Tr "packages.details.author"}}">{{svg "octicon-person" 16 "mr-3"}} {{.PackageDescriptor.Metadata.Maintainer}}</div>{{end}} |
||||||
|
{{if .PackageDescriptor.Metadata.ProjectURL}}<div class="item">{{svg "octicon-link-external" 16 "mr-3"}} <a href="{{.PackageDescriptor.Metadata.ProjectURL}}" target="_blank" rel="noopener noreferrer me">{{.locale.Tr "packages.details.project_site"}}</a></div>{{end}} |
||||||
|
{{end}} |
||||||
@ -0,0 +1,252 @@ |
|||||||
|
// Copyright 2023 The Gitea Authors. All rights reserved.
|
||||||
|
// SPDX-License-Identifier: MIT
|
||||||
|
|
||||||
|
package integration |
||||||
|
|
||||||
|
import ( |
||||||
|
"archive/tar" |
||||||
|
"bytes" |
||||||
|
"compress/gzip" |
||||||
|
"fmt" |
||||||
|
"io" |
||||||
|
"net/http" |
||||||
|
"strings" |
||||||
|
"testing" |
||||||
|
|
||||||
|
"code.gitea.io/gitea/models/db" |
||||||
|
"code.gitea.io/gitea/models/packages" |
||||||
|
"code.gitea.io/gitea/models/unittest" |
||||||
|
user_model "code.gitea.io/gitea/models/user" |
||||||
|
"code.gitea.io/gitea/modules/base" |
||||||
|
debian_module "code.gitea.io/gitea/modules/packages/debian" |
||||||
|
"code.gitea.io/gitea/tests" |
||||||
|
|
||||||
|
"github.com/blakesmith/ar" |
||||||
|
"github.com/stretchr/testify/assert" |
||||||
|
) |
||||||
|
|
||||||
|
func TestPackageDebian(t *testing.T) { |
||||||
|
defer tests.PrepareTestEnv(t)() |
||||||
|
user := unittest.AssertExistsAndLoadBean(t, &user_model.User{ID: 2}) |
||||||
|
|
||||||
|
packageName := "gitea" |
||||||
|
packageVersion := "1.0.3" |
||||||
|
packageDescription := "Package Description" |
||||||
|
|
||||||
|
createArchive := func(name, version, architecture string) io.Reader { |
||||||
|
var cbuf bytes.Buffer |
||||||
|
zw := gzip.NewWriter(&cbuf) |
||||||
|
tw := tar.NewWriter(zw) |
||||||
|
tw.WriteHeader(&tar.Header{ |
||||||
|
Name: "control", |
||||||
|
Mode: 0o600, |
||||||
|
Size: 50, |
||||||
|
}) |
||||||
|
fmt.Fprintf(tw, "Package: %s\nVersion: %s\nArchitecture: %s\nDescription: %s\n", name, version, architecture, packageDescription) |
||||||
|
tw.Close() |
||||||
|
zw.Close() |
||||||
|
|
||||||
|
var buf bytes.Buffer |
||||||
|
aw := ar.NewWriter(&buf) |
||||||
|
aw.WriteGlobalHeader() |
||||||
|
hdr := &ar.Header{ |
||||||
|
Name: "control.tar.gz", |
||||||
|
Mode: 0o600, |
||||||
|
Size: int64(cbuf.Len()), |
||||||
|
} |
||||||
|
aw.WriteHeader(hdr) |
||||||
|
aw.Write(cbuf.Bytes()) |
||||||
|
return &buf |
||||||
|
} |
||||||
|
|
||||||
|
distributions := []string{"test", "gitea"} |
||||||
|
components := []string{"main", "stable"} |
||||||
|
architectures := []string{"all", "amd64"} |
||||||
|
|
||||||
|
rootURL := fmt.Sprintf("/api/packages/%s/debian", user.Name) |
||||||
|
|
||||||
|
t.Run("RepositoryKey", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
req := NewRequest(t, "GET", rootURL+"/repository.key") |
||||||
|
resp := MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Equal(t, "application/pgp-keys", resp.Header().Get("Content-Type")) |
||||||
|
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP PUBLIC KEY BLOCK-----") |
||||||
|
}) |
||||||
|
|
||||||
|
for _, distribution := range distributions { |
||||||
|
t.Run(fmt.Sprintf("[Distribution:%s]", distribution), func(t *testing.T) { |
||||||
|
for _, component := range components { |
||||||
|
for _, architecture := range architectures { |
||||||
|
t.Run(fmt.Sprintf("[Component:%s,Architecture:%s]", component, architecture), func(t *testing.T) { |
||||||
|
t.Run("Upload", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
uploadURL := fmt.Sprintf("%s/pool/%s/%s/upload", rootURL, distribution, component) |
||||||
|
|
||||||
|
req := NewRequestWithBody(t, "PUT", uploadURL, bytes.NewReader([]byte{})) |
||||||
|
MakeRequest(t, req, http.StatusUnauthorized) |
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", uploadURL, bytes.NewReader([]byte{})) |
||||||
|
AddBasicAuthHeader(req, user.Name) |
||||||
|
MakeRequest(t, req, http.StatusBadRequest) |
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", uploadURL, createArchive("", "", "")) |
||||||
|
AddBasicAuthHeader(req, user.Name) |
||||||
|
MakeRequest(t, req, http.StatusBadRequest) |
||||||
|
|
||||||
|
req = NewRequestWithBody(t, "PUT", uploadURL, createArchive(packageName, packageVersion, architecture)) |
||||||
|
AddBasicAuthHeader(req, user.Name) |
||||||
|
MakeRequest(t, req, http.StatusCreated) |
||||||
|
|
||||||
|
pvs, err := packages.GetVersionsByPackageType(db.DefaultContext, user.ID, packages.TypeDebian) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Len(t, pvs, 1) |
||||||
|
|
||||||
|
pd, err := packages.GetPackageDescriptor(db.DefaultContext, pvs[0]) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.Nil(t, pd.SemVer) |
||||||
|
assert.IsType(t, &debian_module.Metadata{}, pd.Metadata) |
||||||
|
assert.Equal(t, packageName, pd.Package.Name) |
||||||
|
assert.Equal(t, packageVersion, pd.Version.Version) |
||||||
|
|
||||||
|
pfs, err := packages.GetFilesByVersionID(db.DefaultContext, pvs[0].ID) |
||||||
|
assert.NoError(t, err) |
||||||
|
assert.NotEmpty(t, pfs) |
||||||
|
assert.Condition(t, func() bool { |
||||||
|
seen := false |
||||||
|
expectedFilename := fmt.Sprintf("%s_%s_%s.deb", packageName, packageVersion, architecture) |
||||||
|
expectedCompositeKey := fmt.Sprintf("%s|%s", distribution, component) |
||||||
|
for _, pf := range pfs { |
||||||
|
if pf.Name == expectedFilename && pf.CompositeKey == expectedCompositeKey { |
||||||
|
if seen { |
||||||
|
return false |
||||||
|
} |
||||||
|
seen = true |
||||||
|
|
||||||
|
assert.True(t, pf.IsLead) |
||||||
|
|
||||||
|
pfps, err := packages.GetProperties(db.DefaultContext, packages.PropertyTypeFile, pf.ID) |
||||||
|
assert.NoError(t, err) |
||||||
|
|
||||||
|
for _, pfp := range pfps { |
||||||
|
switch pfp.Name { |
||||||
|
case debian_module.PropertyDistribution: |
||||||
|
assert.Equal(t, distribution, pfp.Value) |
||||||
|
case debian_module.PropertyComponent: |
||||||
|
assert.Equal(t, component, pfp.Value) |
||||||
|
case debian_module.PropertyArchitecture: |
||||||
|
assert.Equal(t, architecture, pfp.Value) |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
} |
||||||
|
return seen |
||||||
|
}) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("Download", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/pool/%s/%s/%s_%s_%s.deb", rootURL, distribution, component, packageName, packageVersion, architecture)) |
||||||
|
resp := MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Equal(t, "application/vnd.debian.binary-package", resp.Header().Get("Content-Type")) |
||||||
|
}) |
||||||
|
|
||||||
|
t.Run("Packages", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
url := fmt.Sprintf("%s/dists/%s/%s/binary-%s/Packages", rootURL, distribution, component, architecture) |
||||||
|
|
||||||
|
req := NewRequest(t, "GET", url) |
||||||
|
resp := MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
body := resp.Body.String() |
||||||
|
|
||||||
|
assert.Contains(t, body, "Package: "+packageName) |
||||||
|
assert.Contains(t, body, "Version: "+packageVersion) |
||||||
|
assert.Contains(t, body, "Architecture: "+architecture) |
||||||
|
assert.Contains(t, body, fmt.Sprintf("Filename: pool/%s/%s/%s_%s_%s.deb", distribution, component, packageName, packageVersion, architecture)) |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", url+".gz") |
||||||
|
MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", url+".xz") |
||||||
|
MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
url = fmt.Sprintf("%s/dists/%s/%s/%s/by-hash/SHA256/%s", rootURL, distribution, component, architecture, base.EncodeSha256(body)) |
||||||
|
req = NewRequest(t, "GET", url) |
||||||
|
resp = MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Equal(t, body, resp.Body.String()) |
||||||
|
}) |
||||||
|
}) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
t.Run("Release", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release", rootURL, distribution)) |
||||||
|
resp := MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
body := resp.Body.String() |
||||||
|
|
||||||
|
assert.Contains(t, body, "Components: "+strings.Join(components, " ")) |
||||||
|
assert.Contains(t, body, "Architectures: "+strings.Join(architectures, " ")) |
||||||
|
|
||||||
|
for _, component := range components { |
||||||
|
for _, architecture := range architectures { |
||||||
|
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages", component, architecture)) |
||||||
|
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages.gz", component, architecture)) |
||||||
|
assert.Contains(t, body, fmt.Sprintf("%s/binary-%s/Packages.xz", component, architecture)) |
||||||
|
} |
||||||
|
} |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/by-hash/SHA256/%s", rootURL, distribution, base.EncodeSha256(body))) |
||||||
|
resp = MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Equal(t, body, resp.Body.String()) |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release.gpg", rootURL, distribution)) |
||||||
|
resp = MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP SIGNATURE-----") |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/InRelease", rootURL, distribution)) |
||||||
|
resp = MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
assert.Contains(t, resp.Body.String(), "-----BEGIN PGP SIGNED MESSAGE-----") |
||||||
|
}) |
||||||
|
}) |
||||||
|
} |
||||||
|
|
||||||
|
t.Run("Delete", func(t *testing.T) { |
||||||
|
defer tests.PrintCurrentTest(t)() |
||||||
|
|
||||||
|
distribution := distributions[0] |
||||||
|
architecture := architectures[0] |
||||||
|
|
||||||
|
for _, component := range components { |
||||||
|
req := NewRequest(t, "DELETE", fmt.Sprintf("%s/pool/%s/%s/%s/%s/%s", rootURL, distribution, component, packageName, packageVersion, architecture)) |
||||||
|
MakeRequest(t, req, http.StatusUnauthorized) |
||||||
|
|
||||||
|
req = NewRequest(t, "DELETE", fmt.Sprintf("%s/pool/%s/%s/%s/%s/%s", rootURL, distribution, component, packageName, packageVersion, architecture)) |
||||||
|
AddBasicAuthHeader(req, user.Name) |
||||||
|
MakeRequest(t, req, http.StatusNoContent) |
||||||
|
|
||||||
|
req = NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/%s/binary-%s/Packages", rootURL, distribution, component, architecture)) |
||||||
|
MakeRequest(t, req, http.StatusNotFound) |
||||||
|
} |
||||||
|
|
||||||
|
req := NewRequest(t, "GET", fmt.Sprintf("%s/dists/%s/Release", rootURL, distribution)) |
||||||
|
resp := MakeRequest(t, req, http.StatusOK) |
||||||
|
|
||||||
|
body := resp.Body.String() |
||||||
|
|
||||||
|
assert.Contains(t, body, "Components: "+strings.Join(components, " ")) |
||||||
|
assert.Contains(t, body, "Architectures: "+architectures[1]) |
||||||
|
}) |
||||||
|
} |
||||||
|
After Width: | Height: | Size: 4.2 KiB |
Loading…
Reference in new issue