Summary:
This addresses three issues with the current patch management system:
# Two people developing at the same time often pick the same SQL patch number, and then have to go rename it. The system catches this, but it's silly.
# Second/third-party developers can't use the same system to manage auxiliary storage they may want to add.
# There's no way to build mock databases for unit tests that need to do reads.
To resolve these things, you can now name your patches whatever you want and conflicts are just merge conflicts, which are less of a pain to fix than filename conflicts.
Dependencies are now a DAG, with implicit dependencies created on the prior patch if no dependencies are specified. Developers can add new concrete subclasses of `PhabricatorSQLPatchList` to add storage management, and define the dependency branchpoint of their patches so they apply in the correct order (although, generally, they should not depend on the mainline patches, presumably).
The commands `storage upgrade --namespace test1234` and `storage destroy --namespace test1234` will allow unit tests to build and destroy MySQL storage.
A "quickstart" mode allows an upgrade from scratch in ~1200ms. Destruction takes about 200ms. These seem like fairily reasonable costs to actually use in tests. Building from scratch patch-by-patch takes about 6000ms.
Test Plan:
- Created new databases from scratch with and without quickstart in a separate test namespace. Pointed the webapp at the test namespaces, browsed around, everything looked good.
- Compared quickstart and no-quickstart dump states, they're identical except for mysqldump timestamps and a few similar things.
- Upgraded a legacy database to the new storage format.
- Destroyed / dumped storage.
Reviewers: edward, vrana, btrahan, jungejason
Reviewed By: btrahan
CC: aran, nh
Maniphest Tasks: T140, T345
Differential Revision: https://secure.phabricator.com/D2323
Summary: I'm not aware of an easy way to get this information through normal tools. I'm sure there's some fancy GUI client that has it but this seems worth keeping around.
Test Plan: ran script, got helpful information about data sizes
Reviewers: nh, btrahan
Reviewed By: nh
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1916
Summary: This is the script used for conversion: P319
Test Plan:
Update diff with UTF-8 characters in description.
`sql/upgrade_schema.php`
Verify data in DB and that it looks good on web.
Reviewers: epriestley, nh
Reviewed By: epriestley
CC: aran, epriestley
Maniphest Tasks: T327
Differential Revision: https://secure.phabricator.com/D1830
Summary:
1) Dry run option allows you to run the script to see what patches would get
applied (without actually applying them).
2) Max version allows you to do a partial upgrade.
Task ID: #
Blame Rev:
Test Plan:
ran a dry run twice (to make sure it didn't do anything) and no max version
ran with a dry run and max version to check max version logic works correctly
ran with just a max version, and only patches up to that point got applied
Revert Plan:
Tags:
Reviewers: epriestley, btrahan, jungejason
CC: aran, epriestley
Differential Revision: https://secure.phabricator.com/D1758
Summary: There are currently two files, but all scripts require both of them,
which is clearly silly. In the longer term I want to rewrite all of this init
stuff to be more structured (e.g., merge webroot/index.php and __init_script__
better) but this reduces the surface area of the ad-hoc "include files" API we
have now, at least.
Test Plan:
- Grepped for __init_env__.php (no hits)
- Ran a unit test (to test unit changes)
- Ran a daemon (to test daemon changes)
Reviewers: jungejason, nh, tuomaspelkonen, aran
Reviewed By: jungejason
CC: aran, jungejason
Differential Revision: 976
Summary:
A bunch of the .sql patch files don't explicitly specify the table engine, but
we should always use InnoDB with the exception of one table which needs MyISAM
for FULLTEXT.
MySQL doesn't no-op an ALTER TABLE statment that changes the engine back to
itself and converting large tables can be time consuming, so convert only the
required tables.
Test Plan: Ran on secure.phabricator.com and my local box, it fixed all the
issues in about 3 seconds on secure.phabricator.com and <<1 second on my local.
Reviewed By: codeblock
Reviewers: codeblock, tuomaspelkonen, jungejason, aran
CC: aran, epriestley, codeblock
Differential Revision: 641
Summary:
remove accessing the db config info directly. Use
DatabaseConfigurationProvider instead. Also fixed a minor issue where
different number of newlines are output in PhabricatorSetup.php's output.
Test Plan:
executed upgrade_schema.php; executed PhabricatorSetup.php by
setting 'phabricator.setup' to true.
Reviewed By: epriestley
Reviewers: epriestley
CC: aran, jungejason, epriestley
Differential Revision: 443
Summary:
This already pretty much works, document it explicitly.
Test Plan:
Moved my MySQL server over to port 3307.
Reviewed By: aran
Reviewers: jungejason, aran
CC: aran
Differential Revision: 411
Summary: Normally this gives you a prompt about taking down services, provide a noninteractive mode for scripting the upgrade process.
Also drop a generally bad/confusing/irrelevant piece of advice from the documentation and replace it with information about -f.
Test Plan: Ran with and without -f. Ran with -h.
Reviewers: moskov, tuomaspelkonen, jungejason, aran
CC:
Differential Revision: 387
Summary:
Alters the installation instructions to guide installers into a "setup" mode
which does config file sanity checking.
Test Plan:
Put myself in setup mode, simulated all the failures it detects, took myself out
of setup mode, Phabricator works OK.
Reviewed By: tuomaspelkonen
Reviewers: jungejason, tuomaspelkonen, aran
CC: aran, tuomaspelkonen, epriestley
Differential Revision: 230
Summary:
In a basically reasonable configuration where you connect
with a non-privileged user from the web workflow, upgrade_schema.php
won't have enough privileges. Allow the user to override the normal
auth with -u and -p.
Test Plan:
Tried to do a schema upgrade with an underprivileged user,
got a useful error message instead of garbage.
Reviewed By: Girish
Reviewers: Girish, davidrecordon, jungejason, tuomaspelkonen, aran
CC: aran, epriestley, Girish
Differential Revision: 191
Summary:
This global 'const' syntax was introduced in PHP 5.3:
http://www.php.net/manual/en/language.constants.syntax.php
We're PHP 5.2.x elsewhere so just use define(). Made the constant a little more
specific too.
Test Plan:
Ran upgrade_schema.php script.
Reviewed By: Girish
Reviewers: tuomaspelkonen, Girish, davidrecordon
CC: jungejason, aran, epriestley, Girish
Differential Revision: 190
Summary: While I should fix the transactional stuff, that patch is going to be
tricky and transactions have some performance implications. This is a simple
fix which prevents the race.
Instead of having the data point at the event ID, have the event point at a
data ID. Insert the data first, then insert the event with the right data
pointer. This is super simple and prevents the race issue.
Test Plan:
- Ran the schema upgrade script, verified that the database was
correctly upgraded. Was also prompted to stop daemons.
- Ran 'repository-launch-master', verified that the discovery daemons were
able to discover new commits and insert events for them. Verified the
committask daemon was consuming events and converting them into tasks.
- Verified new tasks looked correct in the database.
- Browsed web interface.
Reviewers: jungejason
CC: tuomaspelkonen
Differential Revision: 133
Summary:
Created a script the runs all the necessary patches for db schema.
Stores information in the db about the latest patch that was applied.
Test Plan:
Created two test files '024.test.sql' and '023.test.sql' in this order.
'023' creates a database and '024' creates a table in this db. First ran
'./upgrade_schema.php 23' and made sure that patches were applied in order.
Then ran './upgrade_schema.php' to make sure db was up-to-date. Checked
manually from the db that the database and table exists.
Reviewed By: epriestley
Reviewers: epriestley
CC: jungejason, epriestley, tuomaspelkonen
Differential Revision: 115
