ok op@

This is an initial barebones implementation which provides the absolute minimum of functionality required to serve got(1) and git(1) clients. Basic fetch/send functionality has been tested and seems to work here, but this server is not yet expected to be stable. More testing is welcome. See the man pages for setup instructions. The current design uses one reader and one writer process per repository, which will have to be extended to N readers and N writers in the future. At startup, each process will chroot(2) into its assigned repository. This works because gotd(8) can only be started as root, and will then fork+exec, chroot, and privdrop. At present the parent process runs with the following pledge(2) promises: "stdio rpath wpath cpath proc getpw sendfd recvfd fattr flock unix unveil" The parent is the only process able to modify the repository in a way that becomes visible to Git clients. The parent uses unveil(2) to restrict its view of the filesystem to /tmp and the repositories listed in the configuration file gotd.conf(5). Per-repository chroot(2) processes use "stdio rpath sendfd recvfd". The writer defers to the parent for modifying references in the repository to point at newly uploaded commits. The reader is fine without such help, because Git repositories can be read without having to create any lock-files. gotd(8) requires a dedicated user ID, which should own repositories on the filesystem, and a separate secondary group, which should not have filesystem-level repository access, and must be allowed access to the gotd(8) socket. To obtain Git repository access, users must be members of this secondary group, and must have their login shell set to gotsh(1). gotsh(1) connects to the gotd(8) socket and speaks Git-protocol towards the client on the other end of the SSH connection. gotsh(1) is not an interactive command shell. At present, authenticated clients are granted read/write access to all repositories and all references (except for the "refs/got/" and the "refs/remotes/" namespaces, which are already being protected from modification). While complicated access control mechanism are not a design goal, making it possible to safely offer anonymous Git repository access over ssh(1) is on the road map.

Needed by future gotd(8), where progress reports will be sent to a network socket, rather than a local terminal.

Needed by future gotd(8). ok op@

Future gotd(8) needs to run got_pack_create() in a chroot environment, so we can no longer open new temporary files inside got_pack_create(). ok op@

The old code required a seekable output file. This conflicts with requirements of future gotd(8), which will write pack file data to network sockets. ok op@

ok stsp@

ok op, tracey

Throwing this into the mix for a while, we can always get rid of it again if it becomes annoying. No objections from stsp@

ok stsp@

This allows the main process to avoid looping over all object IDs again in case the pack file used for enumeration is complete. ok op@

The problem that was found in the earlier version has been fixed. ok op@

This implementation marked commits and trees as enumerated before all trees which they depend on were enumerated. This behaviour leads to incomplete pack files when a tree is only partially packed and got-read-pack hits a missing tree entry as a result. The algorithm must be reworked such that packed leave nodes are marked enumerated first, then bubble-up. Found by op@

pointed out by op@

ok op@

The 'nskip' variable is supposed to reflect commits which are waiting on the queue and have the 'skip' color. Only increment 'nskip' when adding such commits to the queue. Problem observed with got send -T and a tag pointing to a deleted branch. Test to reproduce the bug written by op@.

change the three hardcoded seeds to fresh ones generated on demand via arc4random. Suggested/fixed by and ok stsp@

This flag can be expressed as m->reused_delta_offset != 0 because all deltas in valid pack files will be written at a non-zero offset. We allocate a huge number of these structs during packing, so every little bit helps.

With files sorted properly for deltification we produce better deltas but end up consuming more memory and risk running into OpenBSD ulimits during packing. To compensate, reduce the threshold for the amount of delta data we store in memory, spooling more deltas into the cache file. ok op@

This reduces memory use by gotadmin pack. The goal is to sort files which share a path next to each other for deltification. A hash of the path is good enough for this purpose and consumes less memory than a verbatim copy of the path. Git does something similar. ok op@

The old code was broken and stored an empty path or filenames, instead of a repository-relative path. Which means we didn't sort files for deltification as was intended. Fixing this provides much better deltas in large pack files written by gotadmin pack -a. In my test case, pack size changed from 2GB to 1.5GB. ok op@