* [vcpkg] Implement --x-write-nuget-packages-config= setting for `install` and `x-set-installed`.
* [vcpkg] Add end-to-end testing suite for install, remove, and binary caching
* [vcpkg] Define `$TestingRoot in end-to-end-tests.ps1
* [vcpkg] Address CR comments
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Upgrade find_acquire_program(NUGET) to 5.5.1. Add partial SHA to temporary download paths to avoid collisions.
* [vcpkg] Fix regression in `vcpkg export --nuget` due to file locking on .vcpkg-root
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
==== Changes Related to manifests ====
* Add the `manifests` feature flag
* This only says whether we look for a `vcpkg.json` in the cwd, not
whether we support parsing manifests (for ports, for example)
* Changes to the manifests RFC
* `"authors"` -> `"maintainers"`
* `--x-classic-mode` -> `-manifests` \in `vcpkg_feature_flags`
* reserve `"core"` in addition to `"default"`, since that's already
reserved for features
* Add a small helper note about what identifiers must look like
* `<license-string>`: SPDX v3.8 -> v3.9
* `"feature"."description"` is allowed to be an array of strings as well
* `"version"` -> `"version-string"` for forward-compat with versions
RFC
* Add the `--feature-flags` option
* Add the ability to turn off feature flags via passing
`-<feature-flag>` to `VCPKG_FEATURE_FLAGS` or `--feature-flags`
* Add CMake toolchain support for manifests
* Requires either:
* a feature flag of `manifests` in either `Env{VCPKG_FEATURE_FLAGS}`
or `VCPKG_FEATURE_FLAGS`
* Passing the `VCPKG_ENABLE_MANIFESTS` option
* The toolchain will install your packages to
`${VCPKG_MANIFEST_DIR}/vcpkg_installed`.
* Add MSBuild `vcpkg integrate install` support for manifests
* Requires `VcpkgEnableManifest` to be true
* `vcpkg create` creates a port that has a `vcpkg.json` instead of a
`CONTROL`
* argparse, abseil, 3fd, and avisynthplus ports switched to manifest
from CONTROL
* Add support for `--x-manifest-root`, as well as code for finding it if
not passed
* Add support for parsing manifests!
* Add a filesystem lock!
==== Important Changes which are somewhat unrelated to manifests ====
* Rename `logicexpression.{h,cpp}` to `platform-expression.{h,cpp}`
* Add `PlatformExpression` type which takes the place of the old logic
expression
* Split the parsing of platform expressions from checking whether
they're true or not
* Eagerly parse PlatformExpressions as opposed to leaving them as
strings
* Add checking for feature flag consistency
* i.e., if `-binarycaching` is passed, you shouldn't be passing
`--binarysource`
* Add the `Json::Reader` type which, with the help of user-defined
visitors, converts JSON to your internal type
* VcpkgArgParser: place the switch names into a constant as opposed to
using magic constants
* In general update the parsing code so that this ^ works
* Add `Port-Version` fields to CONTROL files
* This replaces the existing practice of
`Version: <my-version>-<port-version>`
==== Smaller changes ====
* small drive-by cleanups to some CMake
* `${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}` ->
`${CURRENT_INSTALLED_DIR}`
* Remove `-analyze` when compiling with clang-cl, since that's not a
supported flag (vcpkg's build system)
* Add a message about which compiler is detected by vcpkg's build
system machinery
* Fix `Expected::then`
* Convert `""` to `{}` for `std::string` and `fs::path`, to avoid a
`strlen` (additionally, `.empty()` instead of `== ""`, and `.clear()`)
* Add `Strings::strto` which converts strings to numeric types
* Support built-in arrays and `StringView` for `Strings::join`
* Add `operator<` and friends to `StringView`
* Add `substr` to `StringView`
* SourceParagraphParser gets some new errors
* Introduce buffered_print class to manage buffered write patterns like in the ci command.
* Remove --purge-tombstones option.
* Law of demeter.
* Make buffered_print imobile.
* buffered_print => BufferedPrint
* Fix merge conflict.
* [vcpkg] Initial implementation of --x-binarysource=nuget
* [vcpkg] Remove double-double quoting of CMake arguments
* [vcpkg] Update nuget.exe to 5.5.1 to support Azure DevOps Artifacts
* [vcpkg] Enable batching of NuGet server calls with prefetch(). Add `interactive` binarysource.
* [vcpkg] Add `nugetconfig` binary source
* [vcpkg] Short circuit querying remote NuGet servers once all refs are found
* [vcpkg] Add experimental help for binary caching
* [vcpkg] Improved NuGet cache package descriptions and version formatting
* [vcpkg] Rename `CmdLineBuilder::build()` to extract()
* [vcpkg-help] Ascii-betize help topics
* [vcpkg] Add tests for cmdlinebuilder. Improve handling of quotes and slashes.
* [vcpkg] Addressing code review comments
* [vcpkg] Add tests for vcpkg::reformat_version()
* [vcpkg] Added test for vcpkg::generate_nuspec()
* [vcpkg] Add tests for vcpkg::XmlSerializer
* [vcpkg] Addressed code review comment
* [vcpkg] Add test for vcpkg::Strings::find_first_of
* [vcpkg] Fix machine-specific paths in test for vcpkg::generate_nuspec()
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Use XDG Base Directory Specification on non-Windows
* [vcpkg] Move user-wide binary cache on Windows to $LOCALAPPDATA/vcpkg/archives
* [vcpkg] Address code review comments; refactor other uses of LOCALAPPDATA
* [vcpkg] Address code review comments
* [vcpkg] filesystem::path::append() accepts string arguments, not paths.
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Refactor out abi_tags_from_pre_build_info()
* [vcpkg] Track Windows toolchain file in triplet hash
* [vcpkg] Improve error messages when constructing PreBuildInfo
* [vcpkg] Extract InstallPlanAction::BuildAbiInfo
* [vcpkg] Extract Build::EnvCache and private-impl VcpkgPaths
* [vcpkg] Enable compiler hash detection when binarycaching is enabled
* [vcpkg] Downgrade warning about missing ABI keys
When binarycaching is not enabled, this warning is spurious and provides no user value.
* [vcpkg] Cleanup
* [vcpkg] Refactor compiler tracking into triplet abi computation
Move several static global caches into VcpkgPaths/EnvCache.
Add feature flag 'compilertracking' to enable the new feature.
* [vcpkg] Refactor out PreBuildInfo::using_vcvars()
Move VcpkgTripletVar into build.cpp because it is not used outside that file.
* [vcpkg] Address some code analysis warnings
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Fix issue #9781 by exporting from the installed directory
* [vcpkg] Address code review comments for #11015
* [vcpkg] Remove duplicate triplet in installed path from exports
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg metrics] start using json library
Additionally, add floats to the JSON library since they're required.
* [vcpkg metrics] allow users to disable metrics after the build
Additionally, as a drive by, fix UUID generation
* fix metrics data
* code review
`files.h/files.cpp`:
* Add end and else comments to all macros.
* Add "remove_all_inside" function which empties a directory without actually deleting the directory. This is necessary to handle the case where the directory is actually a directory symlink.
* Change remove_all to use std::remove when VCPKG_USE_STD_FILESYSTEM is set; this will engage POSIX delete support available on current Win10.
`commands.ci.cpp`: empty "installed".
`*/initialize_environment.*`: No longer clean the directories outside the tool.
`ci-step.ps1`: Remove unused console output tee-ing.
* [vcpkg] Add initial JSON support
This adds a JSON parser, as well as the amount of unicode support
required for JSON parsing to work according to the specification. In the
future, I hope to rewrite our existing XML files into JSON.
Additionally, as a drive-by, we've added the following:
* add /wd4800 to pragmas.h -- this is a "performance warning", for when
you implicitly convert pointers or integers to bool, and shouldn't be
an issue for us.
* Switched Parse::ParserBase to read unicode (as utf-8), as opposed to
ASCII
* Building again under VCPKG_DEVELOPMENT_WARNINGS, yay!
This command takes a list of ports, and causes the final state of the
installed directory to be as-if one ran the install on an empty
installed directory (removing any unnecessary packages).
This is especially useful with the new `--x-install-root` option, which
allows one to set the `installed` directory for vcpkg to use.
Additionally, as a drive-by, we do some `stdfs` clean-up and add a
`.is_feature()` member function to BinaryParagraph (as opposed to
checking for `.feature().empty()`, which is far less clear to read).
This feature is experimental.
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
* `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
of {gcc, clang, msvc}
* `vcpkg_detect_standard_library` -- get the name of the standard
library we're linking to, as one of {libstdc++, libc++, msvc-stl}
* `vcpkg_detect_std_filesystem` -- figure out how to link and call
into C++17's filesystem; whether one needs to link to `stdc++fs` or
`c++fs`, and whether to use `<filesystem>` or
`<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
`VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
* Renamed to `VCPKG_USE_STD_FILESYSTEM`
Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.
And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
* [vcpkg] Correct UInt128 code 😇
`UInt128::operator<<(x, y)` should clear the bottom 64 bits of `x` if
`y >= 64`; however, we don't do this, and so we duplicate `x`'s bottom
bits into `x.top` instead of moving them. Similarly, we have the
opposite problem for `UInt128::operator>>`. This commit fixes these
latent bugs, which we weren't hitting because the thing we use them for
never actually shifts more than 64 bits.
* [vcpkg] Rewrite process spawning on windows to always use CreateProcess, enabling better environment handling
* [vcpkg] Use environment cache to avoid calling vcvars multiple times
* [vcpkg] Increase buffer size while computing hashes
* [vcpkg] Remove unneeded cmd.exe wrapper process on all CreateProcess calls
* [vcpkg] Fix .vcxproj{,.filters}
* [vcpkg] Upgrade Ctrl-C state machine to handle multiple background processes.
* [vcpkg] Fix regression while launching metrics: 'start' can't be passed directly to CreateProcessW
* [vcpkg] Fix typo on non-Windows
* [vcpkg] Fix various uses of >NUL across the code
* [vcpkg] Fix various uses of >NUL across the code
* [vcpkg] Consolidate several internal parsers together (packagespecs + logicexpression + control) and enhance error messages
* [vcpkg] Migrate Build-Depends parsing to new framework
* [vcpkg] Fix tests. Re-enable underscores in feature names due to libwebp[vwebp_sdl] -- todo: rename this feature and remove underscores.
* remove unfinished "supports" tag
* extract "supports" from control files
But do nothing with the value
* Start `Supports` documentation
* Use Supports in a bunch of control files
I only tried matching the already existing logic in the portfile.cmake.
* Cmake var provider (#8)
* Cmake var provider (#9)
* fix windows build (#10)
* Add missing files to build
* Fix test (#11)
* adding hooks for cmake variables in expressions
* Adding hooks for 'supports' in CI test
* Fix test (#12)
* Add overrides to evaluation environment
* use "supported" tag in CI testing
* cleanup comment
* Fix issues with PR
* [var_provider] Get library linkage variables from triplet
* Fix compilation errors in tests
* Add unimplemented functions
* Fix unit tests part 1
* Fix issue when buildtrees dir does not exist
* Change binary output hash
* Fix handling of * feature
* Add core feature when using *
* Do not add Default-Features when installing 'core'
* [vcpkg] WIP. 6 failing tests.
* [vcpkg] WIP. 1 failing tests.
* [vcpkg] WIP. 0 failing tests.
* [vcpkg] Removed 'remove_graph'. 0 failing tests.
* [vcpkg] Removed 'install_graph'. 0 failing tests.
* [vcpkg] Remove AnyAction; replace with ActionPlan
* [vcpkg] Minor cleanup.
* [vcpkg][z3][qt5-connectivity][qt5-purchasing] Improve error messages while parsing. Fix a few trivial port issues.
* [vcpkg] Work around ICE with MSVC v140
* [vcpkg] Add purge on fail to decompress for CI
* [vcpkg] Fix parsing of nested parentheses in qualifiers
* [vcpkg] Fix Linux builds (explicit qualification in declaration)
* [vcpkg] Fix Build-Depends implying default features. Fix qualified dependencies regression.
* [mmx] Add to skip list and full rebuild -- mmx causes problems by installing 'sched.h'
* [libpqxx][mqtt-cpp] Prevent installing include/CMakeLists.txt
* [cppitertools] Fix installed include namespace (should be include/cppitertools)
* [libsoundio] Move headers into soundio/ subdirectory as per original cmake
* [ci.baseline] Temporarily skip charls due to conflict with dcmtk
* [vcpkg] Add restricted include files post build check -- bump global abi version
* [libsoundio] Hotfix stray line in portfile
* [vcpkg] Fix regression: CMake information was not being displayed for build-and-install actions
* [jsonnet] Fix installation of internal headers; use system nlohmann-json
* [grpc][upb] Teach grpc to use packaged upb. Add find_package(upb). Remove inappropriate upb features.
* [zfp] Move problematic 'include/bitstream.h' to 'include/zfp/bitstream.h'
* [x265] Bump control version to trigger rebuild after zfp conflict
* [akali] Disable parallel configure
* [dirent][dlfcn-win32][getopt-win32][pthreads] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS
* [ci.baseline] Update baseline for improved upb support
* [tgui] Disable parallel configure
* [libiconv] Enable VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS
* [aws-sdk-cpp] Disable parallel configure
* [vcpkg] Implement policy VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS
* [aws-sdk-cpp] Fix amount of escaping semicolons -- Note: I do not know the root cause requiring this change
* [libodb-sqlite] Fix configuring into source directory
* [gettext] Grandfather into VCPKG_POLICY_ALLOW_RESTRICTED_HEADERS
* [libodb] DISABLE_PARALLEL_CONFIGURE
* [vcpkg] Add 'config.h' and 'local.h' to restricted header list
* [mcpp] Remove unused and problematic include 'config.h' from installed files
* [teemo] Move installed headers into subdirectory to prevent conflicts with x265
* [ci.baseline] Update current OSX. Skip libmesh on all platforms due to heavy conflicts.
* [vcpkg] Add 'slice.h' as a restricted header
* [osg] Improve accuracy of dependencies (disable some, add some to Depends)
* [vcpkg] Skip invoking a subprocess for 0 specs in load_tag_vars
* [ci.baseline] Skip mongo-c-driver on osx due to flakiness
* [teemo] Fix incorrect include file read
* [osg] Fix dependency typo: glut -> freeglut
* [vcpkg] Recover some lost performance with the addition of vcpkg_get_tags.
A huge performance cost was loading the triplet files over and over; instead, we splice the sources into a macro and load it once, then just call that macro for each port.
Remove use of hashing because we aren't cross-process-safe anyway (global static will do instead).
* [vcpkg] Change Supports atom 'windows' to include UWP. Improve Supports field documentation.
* [vcpkg] Add docs for VCPKG_ENV_PASSTHROUGH and VCPKG_DEP_INFO_OVERRIDE_VARS
* Fix typo
Co-authored-by: Curtis J Bezault <curtbezault@gmail.com>
Co-authored-by: Victor Romero <romerosanchezv@gmail.com>
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* Add support for building with MinGW
Tested with MSYS2 MinGW 8.3.0, gcc-mcf.lhmouse MinGW 9.2.1,
and StephanTLavavej/mingw-distro!
* Add MinGW toolchain
From your MinGW configured shell you could just use vcpkg to
configure packages.
An x64-mingw triplet would look like:
```
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)
set(VCPKG_ENV_PASSTHROUGH PATH)
set(VCPKG_CMAKE_SYSTEM_NAME MinGW)
```
* Add MinGW community tripplets
x64 tested with https://github.com/StephanTLavavej/mingw-distro
x86, arm64, arm tested with https://github.com/mstorsjo/llvm-mingw
On non-Windows platforms, there is no standard way to get the hash of an
item -- before this PR, what we did was check for the existence of a few
common utility names (shasum, sha1, sha256, sha512), and then call that
utility on a file we created containing the contents we wish to hash.
This PR adds internal hashers for sha1, sha256, and sha512, and
standardizes the interface to allow anyone to implement hashers in the
future.
These hashers are not extremely optimized, so it's likely that in the
future we could get more optimized, but for now we just call out to
BCryptHasher on Windows, since it's standard and easy to use (and about
2x faster for sha1 and sha256, and 1.5x faster for sha512). However,
they are reasonably fast for being unoptimized. I attempted a few minor
optimizations, which actually made the code slower! So as of right now,
it's implemented as just a basic conversion of the code on Wikipedia to
C++. I have tested these on the standard NIST test vectors (and those
test vectors are located in vcpkg-test/hash.cpp).
* checkpoint commit
* Only set VCPKG_ENV_OVERRIDES_FILE if it exists
* First pass at working port-toolchain
* Update VERSION.txt
* Return rvalue
* Fix compilation error
* Some fixes are requested by @ubsan
* Fix another compilation error
- Add the "VCPKG_DEVELOPMENT_WARNINGS" flag
- setting "WERROR" will also set this flag
- This flag is set by default
- on GCC/clang, this will pass '-Wall -Wextra -Wpedantic -Werror'
- on GCC, this will additionally pass '-Wmissing-declarations'
- on clang, this will additionally pass '-Wmissing-prototypes'
- on MSVC, this will pass '-W4 -WX'
- On Visual Studio 2017 and later, pass '-permissive-'
- Change the source for fallout of these changes
- add `format` subcommand
- formats all C++ source and header files using clang-format
- move `include/vcpkg-test/catch.h` to `include/catch2/catch.hpp`
- pass CONFIGURE_DEPENDS to file(GLOB)
The VS2015 standard library requires, in debug mode, a comparison
operator on `T × U` and `U × T` to also be a comparison operator on
`T × T` and on `U × U`, and so in
vcpkg::Install::install_package::intersection_compare, I've added two
new `operator()` overloads which take those respectively, on VS2015.
Also, `[nodiscard]` was added to somewhere in `vcpkg/base/strings.h`,
which gives a warning in VS2015 -- thus, I added the `vcpkg/pragmas.h`
include, since that fixes the warning.
I was building under /W3, because CMake hadn't been set up to build
under /W4 -- therefore, I didn't see some warnings.
We also decided to remove the niebloids and instead break ADL by using
`= delete`, since otherwise we get warnings when we define a local
variable with the same name as a niebloid. I also removed `status` and
`symlink_status` from the `files` header, since it's unnecessary now,
and they're just implementation details of `RealFilesystem`.
I also removed some existing uses of unqualified `status(path)`, since
that no longer compiles. I also added `Filesystem::canonical`, to remove
another use of `fs::stdfs` in a function I was already working in.
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
