This changes our PR builds to treat 'fail' in the ci.baseline.txt as 'skip' instead of using tombstones.
We currently have large numbers of spurious failures that get enshrined in PRs through no fault of a PR author, removing the tombstones concept will fix those by allowing the user to retry. This does mean we accept some risk of not detecting when a port is 'fixed', but that failure is reasonable for us to handle after we see it in CI, but that seems worth it given that it lets us get rid of the tombstone concept.
This also helps out the binary caching feature, because we don't have to figure out how to productize tombstones.
* [vcpkg] Implement --x-write-nuget-packages-config= setting for `install` and `x-set-installed`.
* [vcpkg] Add end-to-end testing suite for install, remove, and binary caching
* [vcpkg] Define `$TestingRoot in end-to-end-tests.ps1
* [vcpkg] Address CR comments
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
* [vcpkg] Upgrade find_acquire_program(NUGET) to 5.5.1. Add partial SHA to temporary download paths to avoid collisions.
* [vcpkg] Fix regression in `vcpkg export --nuget` due to file locking on .vcpkg-root
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
==== Changes Related to manifests ====
* Add the `manifests` feature flag
* This only says whether we look for a `vcpkg.json` in the cwd, not
whether we support parsing manifests (for ports, for example)
* Changes to the manifests RFC
* `"authors"` -> `"maintainers"`
* `--x-classic-mode` -> `-manifests` \in `vcpkg_feature_flags`
* reserve `"core"` in addition to `"default"`, since that's already
reserved for features
* Add a small helper note about what identifiers must look like
* `<license-string>`: SPDX v3.8 -> v3.9
* `"feature"."description"` is allowed to be an array of strings as well
* `"version"` -> `"version-string"` for forward-compat with versions
RFC
* Add the `--feature-flags` option
* Add the ability to turn off feature flags via passing
`-<feature-flag>` to `VCPKG_FEATURE_FLAGS` or `--feature-flags`
* Add CMake toolchain support for manifests
* Requires either:
* a feature flag of `manifests` in either `Env{VCPKG_FEATURE_FLAGS}`
or `VCPKG_FEATURE_FLAGS`
* Passing the `VCPKG_ENABLE_MANIFESTS` option
* The toolchain will install your packages to
`${VCPKG_MANIFEST_DIR}/vcpkg_installed`.
* Add MSBuild `vcpkg integrate install` support for manifests
* Requires `VcpkgEnableManifest` to be true
* `vcpkg create` creates a port that has a `vcpkg.json` instead of a
`CONTROL`
* argparse, abseil, 3fd, and avisynthplus ports switched to manifest
from CONTROL
* Add support for `--x-manifest-root`, as well as code for finding it if
not passed
* Add support for parsing manifests!
* Add a filesystem lock!
==== Important Changes which are somewhat unrelated to manifests ====
* Rename `logicexpression.{h,cpp}` to `platform-expression.{h,cpp}`
* Add `PlatformExpression` type which takes the place of the old logic
expression
* Split the parsing of platform expressions from checking whether
they're true or not
* Eagerly parse PlatformExpressions as opposed to leaving them as
strings
* Add checking for feature flag consistency
* i.e., if `-binarycaching` is passed, you shouldn't be passing
`--binarysource`
* Add the `Json::Reader` type which, with the help of user-defined
visitors, converts JSON to your internal type
* VcpkgArgParser: place the switch names into a constant as opposed to
using magic constants
* In general update the parsing code so that this ^ works
* Add `Port-Version` fields to CONTROL files
* This replaces the existing practice of
`Version: <my-version>-<port-version>`
==== Smaller changes ====
* small drive-by cleanups to some CMake
* `${_VCPKG_INSTALLED_DIR}/${VCPKG_TARGET_TRIPLET}` ->
`${CURRENT_INSTALLED_DIR}`
* Remove `-analyze` when compiling with clang-cl, since that's not a
supported flag (vcpkg's build system)
* Add a message about which compiler is detected by vcpkg's build
system machinery
* Fix `Expected::then`
* Convert `""` to `{}` for `std::string` and `fs::path`, to avoid a
`strlen` (additionally, `.empty()` instead of `== ""`, and `.clear()`)
* Add `Strings::strto` which converts strings to numeric types
* Support built-in arrays and `StringView` for `Strings::join`
* Add `operator<` and friends to `StringView`
* Add `substr` to `StringView`
* SourceParagraphParser gets some new errors
* [vcpkg] Fix issue #9781 by exporting from the installed directory
* [vcpkg] Address code review comments for #11015
* [vcpkg] Remove duplicate triplet in installed path from exports
Co-authored-by: Robert Schumacher <roschuma@microsoft.com>
`files.h/files.cpp`:
* Add end and else comments to all macros.
* Add "remove_all_inside" function which empties a directory without actually deleting the directory. This is necessary to handle the case where the directory is actually a directory symlink.
* Change remove_all to use std::remove when VCPKG_USE_STD_FILESYSTEM is set; this will engage POSIX delete support available on current Win10.
`commands.ci.cpp`: empty "installed".
`*/initialize_environment.*`: No longer clean the directories outside the tool.
`ci-step.ps1`: Remove unused console output tee-ing.
This command takes a list of ports, and causes the final state of the
installed directory to be as-if one ran the install on an empty
installed directory (removing any unnecessary packages).
This is especially useful with the new `--x-install-root` option, which
allows one to set the `installed` directory for vcpkg to use.
Additionally, as a drive-by, we do some `stdfs` clean-up and add a
`.is_feature()` member function to BinaryParagraph (as opposed to
checking for `.feature().empty()`, which is far less clear to read).
This feature is experimental.
There are quite a few changes to the CMake build system packaged up into
one set here:
* Added `toolsrc/cmake/utilities.cmake`, which contains the following:
* `vcpkg_detect_compiler` -- get the name of the C++ compiler, as one
of {gcc, clang, msvc}
* `vcpkg_detect_standard_library` -- get the name of the standard
library we're linking to, as one of {libstdc++, libc++, msvc-stl}
* `vcpkg_detect_std_filesystem` -- figure out how to link and call
into C++17's filesystem; whether one needs to link to `stdc++fs` or
`c++fs`, and whether to use `<filesystem>` or
`<experimental/filesystem>`.
* Added a `VCPKG_WARNINGS_AS_ERRORS`, split off from
`VCPKG_DEVELOPMENT_WARNINGS`, which allows one to use the development
warnings without passing -Werror
* Rename `DEFINE_DISABLE_METRICS` to `VCPKG_DISABLE_METRICS` -- the
former will now print a deprecation message and set the latter.
* Now, print a deprecation message on `WERROR`; it doesn't do anything
since the behavior it requested is now the default.
* Pass `-std=c++17` if the compiler allows it, instead of `-std=c++1z`
* Do some code movement
* Pass `USE_STD_FILESYSTEM` if possible, instead of only on minGW
* Renamed to `VCPKG_USE_STD_FILESYSTEM`
Additionally, we now pass `/W4` in Debug mode on x86 in the Visual
Studio build system; this brings it in line with the CMake build system,
and the x64 Visual Studio build system.
And finally, we make some minor code changes to support compiling in
VCPKG_DEVELOPMENT_WARNINGS mode.
* Add support for building with MinGW
Tested with MSYS2 MinGW 8.3.0, gcc-mcf.lhmouse MinGW 9.2.1,
and StephanTLavavej/mingw-distro!
* Add MinGW toolchain
From your MinGW configured shell you could just use vcpkg to
configure packages.
An x64-mingw triplet would look like:
```
set(VCPKG_TARGET_ARCHITECTURE x64)
set(VCPKG_CRT_LINKAGE dynamic)
set(VCPKG_LIBRARY_LINKAGE static)
set(VCPKG_ENV_PASSTHROUGH PATH)
set(VCPKG_CMAKE_SYSTEM_NAME MinGW)
```
* Add MinGW community tripplets
x64 tested with https://github.com/StephanTLavavej/mingw-distro
x86, arm64, arm tested with https://github.com/mstorsjo/llvm-mingw
I was building under /W3, because CMake hadn't been set up to build
under /W4 -- therefore, I didn't see some warnings.
We also decided to remove the niebloids and instead break ADL by using
`= delete`, since otherwise we get warnings when we define a local
variable with the same name as a niebloid. I also removed `status` and
`symlink_status` from the `files` header, since it's unnecessary now,
and they're just implementation details of `RealFilesystem`.
I also removed some existing uses of unqualified `status(path)`, since
that no longer compiles. I also added `Filesystem::canonical`, to remove
another use of `fs::stdfs` in a function I was already working in.
I added benchmarks to measure how fast the parallel remove_all code was
-- it turns out, about 3x slower than stdfs::remove_all. Since this was
the case, I removed all of the parallelism and rewrote it serially, and
ended up about 30% faster than stdfs::remove_all (in addition to
supporting symlinks).
In addition, I did the following three orthogonal changes:
- simplified the work queue, basing it on Billy O'Neal's idea
- Fix warnings on older versions of compilers in tests, by splitting
the pragmas out of pch.h.
- Ran clang-format on some files
In fixing up remove_all, the following changes were made:
- On Windows, regular symlinks and directory symlinks are distinct;
as an example, to remove directory symlinks (and junctions, for that
matter), one must use RemoveDirectory. Only on Windows, I added new
`file_type` and `file_status` types, with `file_type` including a new
`directory_symlink` enumerator, and `file_status` being exactly the
same as the old one except using the new `file_type`. On Unix, I
didn't make that change since they don't make a distinction.
- I added new `symlink_status` and `status` functions which use the
new `file_status` on Windows.
- I made `Filesystem::exists` call `fs::exists(status(p))`, as opposed
to the old version which called `stdfs::exists` directly.
- Added benchmarks to `vcpkg-test/files.cpp`. They test the
performance of `remove_all` on small directories (~20 files), with
symlinks and without, and on large directories (~2000 files), with
symlinks and without.
VS 2019 16.3 will contain a couple of source-breaking changes:
* <experimental/filesystem> will be deprecated via an
impossible-to-miss preprocessor "#error The <experimental/filesystem>
header providing std::experimental::filesystem is deprecated by
Microsoft and will be REMOVED. It is superseded by the C++17
<filesystem> header providing std::filesystem. You can define
_SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING to acknowledge
that you have received this warning."
* <filesystem> will no longer include <experimental/filesystem>.
In the long term, I believe that vcpkg should detect when it's being
built with VS 2017 15.7 or newer, compile in C++17 mode, include
<filesystem>, and use std::filesystem. (Activating this for VS 2019 16.0
or newer would also be reasonable.) Similarly for other toolsets
supporting std::filesystem.
In the short term, this commit makes vcpkg compatible with the upcoming
deprecation. First, we need to define the silencing macro before
including the appropriate header. I've chosen to define it
unconditionally (without checking for platform or version), since it
has no effect for other platforms or versions. Second, we need to deal
with <filesystem> no longer including <experimental/filesystem>.
I verified that VS 2015 Update 3 contained <experimental/filesystem>
(back then, it simply included the <filesystem> header, where the
experimental implementation was defined; this was later reorganized).
Therefore, all of vcpkg's supported MSVC toolsets have
<experimental/filesystem>, so we can simply always include it.
I've verified that this builds with both VS 2015 Update 3 and
VS 2019 16.1.3 (the current production version).
* [vcpkg] Modify Filesystem::remove and Filesystem::rename to not throw.
* [.gitignore] Ignore new VS2019 CMake integration default location
* [.gitignore] Ignore CMakeSettings.json in toolsrc
* [vcpkg] Time external processes called with System::cmd_execute
* [vcpkg] Work around VS2019 CMake bug
* [vcpkg] Fix several unused variable warnings.
* [vcpkg] Improve error handling in vcpkg::Files::Filesystem
Always require either std::error_code or LineInfo to print better errors.
* [vcpkg] Fixup missing return value.
Drive by fix: silence warnings in tests.
* [vcpkg] Fix exiting in error_code overload
Drive by fixes for /analyze with VS2019
