mirror of
https://we.phorge.it/source/phorge.git
synced 2024-11-18 12:52:42 +01:00
b5a009337f
Summary: Ref T1049. I don't really want to sink too much time into this right now, but a seemingly reasonable architecture came to me in a dream. Here's a high-level overview of how things fit together: - **"Build"**: In Harbormaster, "build" means any process we want to run against a working copy. It might actually be building an executable, but it might also be running lint, running unit tests, generating documentation, generating symbols, running a deploy, setting up a sandcastle, etc. - `HarbormasterBuildable`: A "buildable" is some piece of code which build operations can run on. Generally, this is either a Differential diff or a Diffusion commit. The Buildable class just wraps those objects and provides a layer of abstraction. Currently, you can manually create a buildable from a commit. In the future, this will be done automatically. - `HarbormasterBuildStep`: A "build step" is an individual build operation, like "run lint", "run unit", "build docs", etc. The step defines how to perform the operation (for example, "run unit tests by executing 'arc unit'"). In this diff, this barely exists. - `HarbormasterBuildPlan`: This glues together build steps into groups or sequences. For example, you might want to "run unit", and then "deploy" if the tests pass. You can create a build plan which says "run step "unit tests", then run step "deploy" on success" or whatever. In the future, these will also contain triggers/conditions ("Automatically run this build plan against every commit") and probably be able to define failure actions ("If this plan fails, send someone an email"). Because build plans will run commands, only administrators can manage them. - `HarbormasterBuild`: This is the concrete result of running a `BuildPlan` against a `Buildable`. It tracks the build status and collects results, so you can see if the build is running/successful/failed. A `Buildable` may have several `Build`s, because you can execute more than one `BuildPlan` against it. For example, you might have a "documentation" build plan which you run continuously against HEAD, but a "unit" build plan which you want to run against every commit. - `HarbormasterBuildTarget`: This is the concrete result of running a `BuildStep` against a `Buildable`. These are children of `Build`. A step might be able to produce multiple targets, but generally this is something like "Unit Tests" or "Lint" and has an overall status, so you can see at a glance that unit tests were fine but lint had some issues. - `HarbormasterBuildItem`: An optional subitem for a target. For lint, this might be an individual file. For unit tests, an individual test. For normal builds, an executable. For deploys, a server. For documentation generation, there might just not be subitems. - `HarbormasterBuildLog`: Provides extra information, like command/execution transcripts. This is where stdout/stderr will get dumped, and general details and other messages. - `HarbormasterBuildArtifact`: Stores side effects or results from build steps. For example, something which builds a binary might put the binary in "Files" and then put its PHID here. Unit tests might put coverage information here. Generally, any build step which produces some high-level output object can use this table to record its existence. This diff implements almost nothing and does nothing useful, but puts most of these object relationships in place. The two major things you can't easily do with these objects are: 1) Run arbitrary cron jobs. Jenkins does this, but it feels tacked on and I don't know of anyone using it for that. We could create fake Buildables to get a similar effect, but if we need to do this I'd rather do it elsewhere in general. Build and cron/service/monitoring feel like pretty different problems to me. 2) Run parameterized/matrix steps (maybe?). Bamboo has this plan/stage/task/job breakdown where a build step can generate a zillion actual jobs, like "build client on x86", "build server on x86", "build client on ARM", "build server on ARM", etc. We can sort of do this by having a Step map to multiple Targets, but I haven't really thought about it too much and it may end up being not-great. I'd guess we have like an 80% chance of getting a clean implementation if/when we get there. I suspect no one actually needs this, or when they do they'll just implement a custom Step and it can be parameterized at that level. I'm not too worried about this overall. The major difference between this and Jenkins/Bamboo/TravisCI is that all three of those are **plan-centric**: the primary object in the system is a build plan, and the dashboard shows you all your build plans and the current status. I don't think this is the right model. One disadvantage is that you basically end up with top-level messaging that says "Trunk is broken", not "Trunk was broken by commit af32f392f". Harbormaster is **buildable-centric**: the primary object in the system is stuff you can run build operations against (commits/branches/revisions), and actual build plans are secondary. The main view will be "recent commits on this branch, and whether they're good or not" -- which I think is what's most important in a larger/more complex product -- not the pass/fail status of all jobs. This also makes it easier and more natural to integrate with Differential and Diffusion, which both care about the overall status of the commit/revision, not the current status of jobs. Test Plan: Poked around, but this doesn't really do anything yet. Reviewers: btrahan Reviewed By: btrahan CC: zeeg, chad, aran, seporaitis Maniphest Tasks: T1049 Differential Revision: https://secure.phabricator.com/D7368
74 lines
3.1 KiB
SQL
74 lines
3.1 KiB
SQL
CREATE TABLE {$NAMESPACE}_harbormaster.harbormaster_buildable (
|
|
id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
|
phid VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
buildablePHID VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
containerPHID VARCHAR(64) COLLATE utf8_bin,
|
|
buildStatus VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
buildableStatus VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
dateCreated INT UNSIGNED NOT NULL,
|
|
dateModified INT UNSIGNED NOT NULL,
|
|
KEY `key_buildable` (buildablePHID),
|
|
KEY `key_container` (containerPHID),
|
|
UNIQUE KEY `key_phid` (phid)
|
|
) ENGINE=InnoDB, COLLATE utf8_general_ci;
|
|
|
|
CREATE TABLE {$NAMESPACE}_harbormaster.harbormaster_buildartifact (
|
|
id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
|
buildablePHID VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
artifactType VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
artifactIndex VARCHAR(12) NOT NULL COLLATE utf8_bin,
|
|
artifactKey VARCHAR(255) NOT NULL COLLATE utf8_bin,
|
|
artifactData LONGTEXT NOT NULL COLLATE utf8_bin,
|
|
dateCreated INT UNSIGNED NOT NULL,
|
|
dateModified INT UNSIGNED NOT NULL,
|
|
UNIQUE KEY `key_artifact` (buildablePHID, artifactType, artifactIndex),
|
|
UNIQUE KEY `key_artifact_type` (artifactType, artifactIndex),
|
|
KEY `key_garbagecollect` (artifactType, dateCreated)
|
|
) ENGINE=InnoDB, COLLATE utf8_general_ci;
|
|
|
|
CREATE TABLE {$NAMESPACE}_harbormaster.harbormaster_buildplan (
|
|
id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
|
phid VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
name VARCHAR(255) NOT NULL,
|
|
planStatus VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
dateCreated INT UNSIGNED NOT NULL,
|
|
dateModified INT UNSIGNED NOT NULL,
|
|
UNIQUE KEY `key_phid` (phid),
|
|
KEY `key_status` (planStatus)
|
|
) ENGINE=InnoDB, COLLATE utf8_general_ci;
|
|
|
|
CREATE TABLE {$NAMESPACE}_harbormaster.harbormaster_buildplantransaction (
|
|
id INT UNSIGNED NOT NULL PRIMARY KEY AUTO_INCREMENT,
|
|
phid VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
authorPHID VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
objectPHID VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
viewPolicy VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
editPolicy VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
commentPHID VARCHAR(64) COLLATE utf8_bin,
|
|
commentVersion INT UNSIGNED NOT NULL,
|
|
transactionType VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
oldValue LONGTEXT NOT NULL COLLATE utf8_bin,
|
|
newValue LONGTEXT NOT NULL COLLATE utf8_bin,
|
|
contentSource LONGTEXT NOT NULL COLLATE utf8_bin,
|
|
metadata LONGTEXT NOT NULL COLLATE utf8_bin,
|
|
dateCreated INT UNSIGNED NOT NULL,
|
|
dateModified INT UNSIGNED NOT NULL,
|
|
|
|
UNIQUE KEY `key_phid` (phid),
|
|
KEY `key_object` (objectPHID)
|
|
|
|
) ENGINE=InnoDB, COLLATE utf8_general_ci;
|
|
|
|
CREATE TABLE {$NAMESPACE}_harbormaster.harbormaster_build (
|
|
id INT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
|
phid VARCHAR(64) NOT NULL COLLATE utf8_bin,
|
|
buildablePHID varchar(64) NOT NULL COLLATE utf8_bin,
|
|
buildPlanPHID varchar(64) NOT NULL COLLATE utf8_bin,
|
|
buildStatus VARCHAR(32) NOT NULL COLLATE utf8_bin,
|
|
dateCreated INT UNSIGNED NOT NULL,
|
|
dateModified INT UNSIGNED NOT NULL,
|
|
UNIQUE KEY `key_phid` (phid),
|
|
KEY `key_buildable` (buildablePHID),
|
|
KEY `key_plan` (buildPlanPHID),
|
|
KEY `key_status` (buildStatus)
|
|
) ENGINE=InnoDB, COLLATE utf8_general_ci;
|