30 Commits

Author SHA1 Message Date
nicole mazzuca
1d8f0acc9c
[vcpkg manifest] Manifest Implementation (#11757)
==== Changes Related to manifests ====

* Add the `manifests` feature flag
  * This only says whether we look for a `vcpkg.json` in the cwd, not
    whether we support parsing manifests (for ports, for example)
* Changes to the manifests RFC
  * `"authors"` -> `"maintainers"`
  * `--x-classic-mode` -> `-manifests` \in `vcpkg_feature_flags`
  * reserve `"core"` in addition to `"default"`, since that's already
    reserved for features
  * Add a small helper note about what identifiers must look like
  * `<license-string>`: SPDX v3.8 -> v3.9
  * `"feature"."description"` is allowed to be an array of strings as well
  * `"version"` -> `"version-string"` for forward-compat with versions
    RFC
* Add the `--feature-flags` option
* Add the ability to turn off feature flags via passing
  `-<feature-flag>` to `VCPKG_FEATURE_FLAGS` or `--feature-flags`
* Add CMake toolchain support for manifests
  * Requires either:
    * a feature flag of `manifests` in either `Env{VCPKG_FEATURE_FLAGS}`
      or `VCPKG_FEATURE_FLAGS`
    * Passing the `VCPKG_ENABLE_MANIFESTS` option
  * The toolchain will install your packages to
    `${VCPKG_MANIFEST_DIR}/vcpkg_installed`.
* Add MSBuild `vcpkg integrate install` support for manifests
  * Requires `VcpkgEnableManifest` to be true
* `vcpkg create` creates a port that has a `vcpkg.json` instead of a
  `CONTROL`
* argparse, abseil, 3fd, and avisynthplus ports switched to manifest
  from CONTROL
* Add support for `--x-manifest-root`, as well as code for finding it if
  not passed
* Add support for parsing manifests!
* Add a filesystem lock!

==== Important Changes which are somewhat unrelated to manifests ====

* Rename `logicexpression.{h,cpp}` to `platform-expression.{h,cpp}`
* Add `PlatformExpression` type which takes the place of the old logic
  expression
  * Split the parsing of platform expressions from checking whether
    they're true or not
  * Eagerly parse PlatformExpressions as opposed to leaving them as
    strings
* Add checking for feature flag consistency
  * i.e., if `-binarycaching` is passed, you shouldn't be passing
    `--binarysource`
* Add the `Json::Reader` type which, with the help of user-defined
  visitors, converts JSON to your internal type
* VcpkgArgParser: place the switch names into a constant as opposed to
  using magic constants
  * In general update the parsing code so that this ^ works
* Add `Port-Version` fields to CONTROL files
  * This replaces the existing practice of
    `Version: <my-version>-<port-version>`

==== Smaller changes ====
* small drive-by cleanups to some CMake
  * `${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}` ->
    `${CURRENT_INSTALLED_DIR}`
  * Remove `-analyze` when compiling with clang-cl, since that's not a
    supported flag (vcpkg's build system)
  * Add a message about which compiler is detected by vcpkg's build
    system machinery
* Fix `Expected::then`
* Convert `""` to `{}` for `std::string` and `fs::path`, to avoid a
  `strlen` (additionally, `.empty()` instead of `== ""`, and `.clear()`)
* Add `Strings::strto` which converts strings to numeric types
* Support built-in arrays and `StringView` for `Strings::join`
* Add `operator<` and friends to `StringView`
* Add `substr` to `StringView`
* SourceParagraphParser gets some new errors
2020-06-30 10:40:18 -07:00
ras0219
aef838536e
[vcpkg] Track compiler information in ABI (#11654)
* [vcpkg] Refactor out abi_tags_from_pre_build_info()

* [vcpkg] Track Windows toolchain file in triplet hash

* [vcpkg] Improve error messages when constructing PreBuildInfo

* [vcpkg] Extract InstallPlanAction::BuildAbiInfo

* [vcpkg] Extract Build::EnvCache and private-impl VcpkgPaths

* [vcpkg] Enable compiler hash detection when binarycaching is enabled

* [vcpkg] Downgrade warning about missing ABI keys

When binarycaching is not enabled, this warning is spurious and provides no user value.

* [vcpkg] Cleanup

* [vcpkg] Refactor compiler tracking into triplet abi computation

Move several static global caches into VcpkgPaths/EnvCache.
Add feature flag 'compilertracking' to enable the new feature.

* [vcpkg] Refactor out PreBuildInfo::using_vcvars()

Move VcpkgTripletVar into build.cpp because it is not used outside that file.

* [vcpkg] Address some code analysis warnings

Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-06-22 14:14:36 -07:00
Billy O'Neal
5504dfa7da
[vcpkg] Turn on tests and PREfast in CI, and fix tests to pass. (#11239)
Co-authored-by: nicole mazzuca <mazzucan@outlook.com>
2020-05-14 12:49:31 -07:00
Robert Schumacher
3d841c9b6a [vcpkg] Compute all ABIs upfront instead of during build_package() 2020-02-11 23:52:56 -08:00
Robert Schumacher
8db6db5dac
[vcpkg] Further parser improvements (#9895)
* [vcpkg] Consolidate several internal parsers together (packagespecs + logicexpression + control) and enhance error messages

* [vcpkg] Migrate Build-Depends parsing to new framework

* [vcpkg] Fix tests. Re-enable underscores in feature names due to libwebp[vwebp_sdl] -- todo: rename this feature and remove underscores.
2020-02-07 11:24:35 -08:00
Phil Christensen
e62d136128
[vcpkg] Add Supports: field. Use contents of triplets instead of names for dependency resolution. (#8601)
* remove unfinished "supports" tag

* extract "supports" from control files
But do nothing with the value

* Start `Supports` documentation

* Use Supports in a bunch of control files

I only tried matching the already existing logic in the portfile.cmake.

* Cmake var provider (#8)

* Cmake var provider (#9)

* fix windows build (#10)

* Add missing files to build

* Fix test (#11)

* adding hooks for cmake variables in expressions

* Adding hooks for 'supports' in CI test

* Fix test (#12)

* Add overrides to evaluation environment

* use "supported" tag in CI testing

* cleanup comment

* Fix issues with PR

* [var_provider] Get library linkage variables from triplet

* Fix compilation errors in tests

* Add unimplemented functions

* Fix unit tests part 1

* Fix issue when buildtrees dir does not exist

* Change binary output hash

* Fix handling of * feature

* Add core feature when using *

* Do not add Default-Features when installing 'core'

* [vcpkg] WIP. 6 failing tests.

* [vcpkg] WIP. 1 failing tests.

* [vcpkg] WIP. 0 failing tests.

* [vcpkg] Removed 'remove_graph'. 0 failing tests.

* [vcpkg] Removed 'install_graph'. 0 failing tests.

* [vcpkg] Remove AnyAction; replace with ActionPlan

* [vcpkg] Minor cleanup.

* [vcpkg][z3][qt5-connectivity][qt5-purchasing] Improve error messages while parsing. Fix a few trivial port issues.

* [vcpkg] Work around ICE with MSVC v140

* [vcpkg] Add purge on fail to decompress for CI

* [vcpkg] Fix parsing of nested parentheses in qualifiers

* [vcpkg] Fix Linux builds (explicit qualification in declaration)

* [vcpkg] Fix Build-Depends implying default features. Fix qualified dependencies regression.

* [mmx] Add to skip list and full rebuild -- mmx causes problems by installing 'sched.h'

* [libpqxx][mqtt-cpp] Prevent installing include/CMakeLists.txt

* [cppitertools] Fix installed include namespace (should be include/cppitertools)

* [libsoundio] Move headers into soundio/ subdirectory as per original cmake

* [ci.baseline] Temporarily skip charls due to conflict with dcmtk

* [vcpkg] Add restricted include files post build check -- bump global abi version

* [libsoundio] Hotfix stray line in portfile

* [vcpkg] Fix regression: CMake information was not being displayed for build-and-install actions

* [jsonnet] Fix installation of internal headers; use system nlohmann-json

* [grpc][upb] Teach grpc to use packaged upb. Add find_package(upb). Remove inappropriate upb features.

* [zfp] Move problematic 'include/bitstream.h' to 'include/zfp/bitstream.h'

* [x265] Bump control version to trigger rebuild after zfp conflict

* [akali] Disable parallel configure

* [dirent][dlfcn-win32][getopt-win32][pthreads] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [ci.baseline] Update baseline for improved upb support

* [tgui] Disable parallel configure

* [libiconv] Enable VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [aws-sdk-cpp] Disable parallel configure

* [vcpkg] Implement policy VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [aws-sdk-cpp] Fix amount of escaping semicolons -- Note: I do not know the root cause requiring this change

* [libodb-sqlite] Fix configuring into source directory

* [gettext] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS

* [libodb] DISABLE_PARALLEL_CONFIGURE

* [vcpkg] Add 'config.h' and 'local.h' to restricted header list

* [mcpp] Remove unused and problematic include 'config.h' from installed files

* [teemo] Move installed headers into subdirectory to prevent conflicts with x265

* [ci.baseline] Update current OSX. Skip libmesh on all platforms due to heavy conflicts.

* [vcpkg] Add 'slice.h' as a restricted header

* [osg] Improve accuracy of dependencies (disable some, add some to Depends)

* [vcpkg] Skip invoking a subprocess for 0 specs in load_tag_vars

* [ci.baseline] Skip mongo-c-driver on osx due to flakiness

* [teemo] Fix incorrect include file read

* [osg] Fix dependency typo: glut -> freeglut

* [vcpkg] Recover some lost performance with the addition of vcpkg_get_tags.

A huge performance cost was loading the triplet files over and over; instead, we splice the sources into a macro and load it once, then just call that macro for each port.
Remove use of hashing because we aren't cross-process-safe anyway (global static will do instead).

* [vcpkg] Change Supports atom 'windows' to include UWP. Improve Supports field documentation.

* [vcpkg] Add docs for VCPKG_ENV_PASSTHROUGH and VCPKG_DEP_INFO_OVERRIDE_VARS

* Fix typo

Co-authored-by: Curtis J Bezault <curtbezault@gmail.com>
Co-authored-by: Victor Romero <romerosanchezv@gmail.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
2020-02-03 14:22:52 -08:00
Nicole Mazzuca
e79f0dc532 [vcpkg] Make Filesystem::remove_all faster #7570
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).

In addition, I did the following three orthogonal changes:
  - simplified the work queue, basing it on Billy O'Neal's idea
  - Fix warnings on older versions of compilers in tests, by splitting
    the pragmas out of pch.h.
  - Ran clang-format on some files

In fixing up remove_all, the following changes were made:
  - On Windows, regular symlinks and directory symlinks are distinct;
    as an example, to remove directory symlinks (and junctions, for that
    matter), one must use RemoveDirectory. Only on Windows, I added new
    `file_type` and `file_status` types, with `file_type` including a new
    `directory_symlink` enumerator, and `file_status` being exactly the
    same as the old one except using the new `file_type`. On Unix, I
    didn't make that change since they don't make a distinction.
  - I added new `symlink_status` and `status` functions which use the
    new `file_status` on Windows.
  - I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
    to the old version which called `stdfs::exists` directly.
  - Added benchmarks to `vcpkg-test/files.cpp`. They test the
    performance of `remove_all` on small directories (~20 files), with
    symlinks and without, and on large directories (~2000 files), with
    symlinks and without.
2019-08-07 16:51:12 -07:00
Victor Romero
9e565e9867
[--overlay-ports] Show location of overriden ports during install plan (#7002)
* [--overlay-ports] Show source location of overlayed ports during install plan

* Code cleanup

* Code cleanup
2019-06-24 12:09:48 -07:00
Victor Romero
f3db66b403
Ports Overlay partial implementation (#6981)
* Ports Overlay feature spec

* Ports Overlay implementation

* [--overlay-ports] Refactor handling of additional paths

* Code cleanup

* [--overlay-ports] Add help

* [depend-info] Support --overlay-ports

* Add method to load all ports using PathsPortFileProvider

* Make PortFileProvider::load_all_control_files() const

* Remove unused code

* [vcpkg] Avoid double-load of source control file between Build::perform_and_exit and Build::perform_and_exit_ex

* [vcpkg] Clang format

* [vcpkg] Fixup build failure introduced in b069ceb2f231

* Report errors from Paragraphs::try_load_port()
2019-06-21 23:50:05 -07:00
Robert Schumacher
39b7876db4 [vcpkg] Randomize topological sort in CI plans to allow concurrent builds to more efficiently interact 2019-01-22 17:11:36 -08:00
Robert Schumacher
a2aeb2f194 [vcpkg] Mark many types noexcept. Make certain code patterns more transparent to /analyze. #ifdef-out unused code on non-windows. 2018-04-12 00:47:17 -07:00
Robert Schumacher
8da8f3e5b3 [vcpkg] Rework dependencies.cpp to improve type safety and error detection 2018-04-11 23:48:32 -07:00
Alexander Karatarakis
3a91b0072d Fix name 2018-03-24 01:28:24 -07:00
Robert Schumacher
eab1d5c531 [vcpkg-ci] Do not rebuild libraries that were previously successful or failed 2018-03-20 19:12:35 -07:00
Robert Schumacher
bbbbfb9e79 [vcpkg] Add tests for create_export_plan and remove unused arguments 2018-02-28 12:15:16 -08:00
Robert Schumacher
8b97ae2dc1 [vcpkg] Fix "just-in-time" requirements calculation 2018-02-25 10:40:28 -08:00
Robert Schumacher
12f19c7a30 [vcpkg] Remove create_install_plan in favor of create_feature_install_plan 2018-02-19 07:11:30 -08:00
Jonathan Hale
425d07ef7f [vcpkg] Implement Default-Features (#2697)
* [vcpkg] Add Default-Feature to make_status_pgh utility function

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Parse "Default-Features" as dependencies and add test for parsing

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Document some methods and structures

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Add install_default_features_test

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Change install_default_features_test to not have preinstalled package

* [vcpkg] Test install behaviour of default features

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Implement default features

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Test default features upgrade behavior

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Implement upgrade with default features

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Test behaviour of upgrade with default features in dependencies

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Make upgrade install new default features

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Move collecting of packages for which to prevent defaults

Further down the line to create_feature_install_plan.

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Fix core missing from default features and potential inf loop

Signed-off-by: Squareys <squareys@googlemail.com>

* [vcpkg] Rename, fix and move some tests

Signed-off-by: Squareys <squareys@googlemail.com>
2018-02-14 16:18:25 -08:00
Alexander Karatarakis
639f993798 Add missing method const 2018-01-23 15:38:08 -08:00
Robert Schumacher
3beeb94ec5 [vcpkg] Use InstalledPackageView instead of unsorted raw vectors 2018-01-23 06:50:24 -08:00
Robert Schumacher
ff8a2d1726 [vcpkg] Remove AnyParagraph and InstallPlanType::INSTALL 2018-01-23 02:21:47 -08:00
Robert Schumacher
0682bb734d [vcpkg] Store Optional<&> instead of Optional<*> 2018-01-22 23:40:41 -08:00
Robert Schumacher
5ac69dd02b [vcpkg] Improve error message upon graph cycle detected. 2017-12-18 23:00:11 -08:00
Robert Schumacher
eb1a7b099e [vcpkg] Fix regressions introduced with update command. Fixed issue with upgrade where it assumed downloads were not allowed. 2017-12-13 05:43:00 -08:00
Robert Schumacher
7a6ffdc75c Revert "[vcpkg-upgrade] Initial commit of upgrade command."
This reverts commit 803347a0c545687f6e6b8b3594b52d11435491b3.
2017-12-13 04:48:13 -08:00
Robert Schumacher
803347a0c5 [vcpkg-upgrade] Initial commit of upgrade command. 2017-12-13 02:22:52 -08:00
Robert Schumacher
925fab565a [vcpkg] --head should only apply to USER_REQUESTED packages. 2017-10-31 03:04:28 -07:00
Alexander Karatarakis
2c9536ce4f [vcpkg ci] Introduce --exclude option 2017-10-26 19:17:12 -07:00
Robert Schumacher
7214c3583b [vcpkg] Push use of UTF-16 to only around Win32 call boundaries. 2017-10-16 11:44:04 -07:00
Robert Schumacher
e17de99599 [vcpkg] Re-layout all files using new organization scheme.
All filenames and directories are lowercase. Use dots for namespace separation.
2017-10-13 18:37:41 -07:00