# Registries — average (https://benchmarks.vlt.io/registries/average) Benchmarks / Registries / average - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, averaged over 2 scenarios (derived, not measured), on GitHub-hosted ubuntu-24.04-arm runners. Every registry is installed from with the same npm client. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline registry.vlt.io is fastest at 14.1s, 1.46× faster than registry.npmjs.org. - Fastest mean install: 14.1s (registry.vlt.io) - Against registry.npmjs.org: 1.46× (faster, on the mean) - Fixtures won: 6 / 6 (where registry.vlt.io was fastest outright) - Spread: 5.7× (between the slowest and fastest tools that completed every fixture) ## Scenario Every tool across all of this benchmark's scenarios at once. Mean of: registry-clean, registry-lockfile. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against registry.npmjs.org | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | registry.vlt.io | vlt | 14.15 | 3.2% | 1.46× faster | −2.9% | 6/6 | — | | 2 | registry.npmjs.org | npm | 20.664 | 5.0% | baseline | +6.8% | 6/6 | — | | 3 | JFrog Artifactory | jfrog | 25.973 | 6.9% | 1.26× slower | −2.6% | 6/6 | — | | 4 | GitHub Packages | github | 42.688 | 5.2% | 2.1× slower | +10.7% | 6/6 | — | | 5 | AWS CodeArtifact | aws | 54.359 | 2.4% | 2.6× slower | +8.7% | 6/6 | — | | 6 | Cloudsmith | cloudsmith | 80.831 | 30.5% | 3.9× slower | +12.8% | 6/6 | — | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | | registry.vlt.io | 14.1s | +1.8% | 2026-09-23 | 2026-10-20 | | registry.npmjs.org | 20.7s | −11.4% | 2026-09-22 | 2026-10-20 | | JFrog Artifactory | 26.0s | −15.6% | 2026-09-22 | 2026-10-20 | | GitHub Packages | 42.7s | −11.8% | 2026-09-22 | 2026-10-20 | | AWS CodeArtifact | 54.4s | −14.3% | 2026-09-22 | 2026-10-20 | | Cloudsmith | 80.8s | +48.3% | 2026-09-23 | 2026-10-20 | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/registries/average/history ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,,23.332,30.778,48.424,63.433, 2026-09-23,13.905,20.595,28.491,41.933,59.156,54.506 2026-09-24,14.526,19.612,27.492,39.879,52.459,68.721 2026-09-25,14.632,23.483,27.884,42.474,54.535,74.999 2026-09-26,15.232,19.306,27.793,37.314,52.04,56.336 2026-09-27,16.806,20.189,29.565,40.671,54.14, 2026-09-28,14.845,20.098,25.402,41.051,52.353, 2026-09-29,14.249,21.08,27.409,45.828,54.631, 2026-09-30,14.592,20.699,24.777,40.707,51.217,47.594 2026-10-01,14.254,20.433,31.558,44.023,56.569,64.556 2026-10-02,15.058,19.494,31.446,45.253,56.122,68.338 2026-10-03,14.903,19.47,27.464,37.491,52.633,60.275 2026-10-04,14.239,19.555,28.96,38.988,53.606,63.247 2026-10-05,13.547,18.133,28.277,40.138,53.204,54.755 2026-10-06,13.975,18.813,26.382,38.23,49.139,48.676 2026-10-07,14.509,19.681,29.477,42.646,55.303,73.562 2026-10-08,14.73,20.161,27.364,41.666,52.869,76.155 2026-10-09,14.644,19.341,29.434,39.834,52.394, 2026-10-10,13.779,18.497,25.362,36.474,49.836,58.666 2026-10-11,15.513,20.216,27.647,39.068,53.067,53.364 2026-10-12,14.942,19.498,26.349,39.813,51.664,51.839 2026-10-13,14.118,20.049,28.112,41.902,52.107,56.593 2026-10-14,15.122,20.231,27.226,42.359,57.66,59.759 2026-10-15,14.21,20.056,27.998,41.636,53.646,64.056 2026-10-16,14.173,18.853,26.829,41.813,53.497,59.628 2026-10-17,15.098,19.671,29.223,39.244,56.179,63.57 2026-10-18,13.934,19.075,28.338,39.861,53.749,60.066 2026-10-19,14.576,19.351,26.655,38.561,50.008,71.659 2026-10-20,14.15,20.664,25.973,42.688,54.359,80.831 ``` #### DNF and coverage - Cloudsmith on Babylon.js: DNF on 5 runs (exit 1): 2026-09-22, 2026-09-27, 2026-09-28, 2026-09-29, 2026-10-09 - Cloudsmith on Large: DNF on 2 runs (exit 1): 2026-09-22, 2026-09-28 - registry.vlt.io on Large: DNF on 1 run (exit 1): 2026-09-22 #### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-10-08: Cloudsmith, +40.9% from its neighbours ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 14.1s | 1.46× faster | | registry.npmjs.org | 20.7s | baseline | | JFrog Artifactory | 26.0s | 1.26× slower | | GitHub Packages | 42.7s | 2.1× slower | | AWS CodeArtifact | 54.4s | 2.6× slower | | Cloudsmith | 80.8s | 3.9× slower | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. #### Astro | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 4.94s | 1.48× faster | | registry.npmjs.org | 7.32s | baseline | | JFrog Artifactory | 10.9s | 1.48× slower | | GitHub Packages | 16.2s | 2.2× slower | | AWS CodeArtifact | 16.7s | 2.3× slower | | Cloudsmith | 19.6s | 2.7× slower | #### Babylon.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 43.6s | 1.34× faster | | registry.npmjs.org | 58.5s | baseline | | JFrog Artifactory | 68.8s | 1.18× slower | | GitHub Packages | 111s | 1.90× slower | | AWS CodeArtifact | 137s | 2.3× slower | | Cloudsmith | 270s | 4.6× slower | #### Large | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 17.4s | 1.59× faster | | registry.npmjs.org | 27.6s | baseline | | JFrog Artifactory | 36.4s | 1.32× slower | | GitHub Packages | 71.8s | 2.6× slower | | AWS CodeArtifact | 95.8s | 3.5× slower | | Cloudsmith | 133s | 4.8× slower | #### Next.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 10.2s | 1.41× faster | | registry.npmjs.org | 14.3s | baseline | | JFrog Artifactory | 18.4s | 1.29× slower | | GitHub Packages | 24.0s | 1.68× slower | | Cloudsmith | 31.2s | 2.2× slower | | AWS CodeArtifact | 31.6s | 2.2× slower | #### Svelte | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 1.50s | 2.1× faster | | registry.npmjs.org | 3.18s | baseline | | Cloudsmith | 3.30s | 1.04× slower | | JFrog Artifactory | 3.91s | 1.23× slower | | GitHub Packages | 6.33s | 1.99× slower | | AWS CodeArtifact | 7.04s | 2.2× slower | #### Vue | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 7.26s | 1.81× faster | | registry.npmjs.org | 13.1s | baseline | | JFrog Artifactory | 17.4s | 1.33× slower | | GitHub Packages | 26.7s | 2.0× slower | | Cloudsmith | 28.3s | 2.2× slower | | AWS CodeArtifact | 38.3s | 2.9× slower | ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | | registry.vlt.io | 4.941 | 43.645 | 17.391 | 10.159 | 1.5 | 7.261 | | registry.npmjs.org | 7.325 | 58.497 | 27.579 | 14.288 | 3.177 | 13.117 | | JFrog Artifactory | 10.861 | 68.787 | 36.432 | 18.401 | 3.913 | 17.441 | | GitHub Packages | 16.234 | 111.181 | 71.755 | 23.973 | 6.331 | 26.654 | | AWS CodeArtifact | 16.69 | 136.684 | 95.83 | 31.592 | 7.041 | 38.315 | | Cloudsmith | 19.634 | 270.071 | 132.542 | 31.188 | 3.296 | 28.256 | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. - Disclosure. vlt publishes this benchmark, and registry.vlt.io is one of the registries it measures. The suite, its configuration and every result file are public. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - Raw results, Astro · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/astro-registry-clean.json - Raw results, Astro · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/astro-registry-lockfile.json - Raw results, Babylon.js · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/babylon-registry-clean.json - Raw results, Babylon.js · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/babylon-registry-lockfile.json - Raw results, Large · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/large-registry-clean.json - Raw results, Large · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/large-registry-lockfile.json - Raw results, Next.js · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/next-registry-clean.json - Raw results, Next.js · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/next-registry-lockfile.json - Raw results, Svelte · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/svelte-registry-clean.json - Raw results, Svelte · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/svelte-registry-lockfile.json - Raw results, Vue · registry-clean (JSON): https://benchmarks-data.vlt.io/2026-10-20/vue-registry-clean.json - Raw results, Vue · registry-lockfile (JSON): https://benchmarks-data.vlt.io/2026-10-20/vue-registry-lockfile.json - This report as JSON: https://benchmarks.vlt.io/api/v1/registries/average - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/registries/average?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Registries, average [Data set]. https://benchmarks.vlt.io/registries/average?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Registries, average}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/registries/average?date=2026-10-20} } ``` # Registries — registry-clean (https://benchmarks.vlt.io/registries/registry-clean) Benchmarks / Registries / registry-clean - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. Every registry is installed from with the same npm client. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline registry.vlt.io is fastest at 17.5s, 1.73× faster than registry.npmjs.org. - Fastest mean install: 17.5s (registry.vlt.io) - Against registry.npmjs.org: 1.73× (faster, on the mean) - Fixtures won: 6 / 6 (where registry.vlt.io was fastest outright) - Spread: 5.5× (between the slowest and fastest tools that completed every fixture) ## Scenario A cold install, holding the package manager constant and varying the registry. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against registry.npmjs.org | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | registry.vlt.io | vlt | 17.473 | 3.3% | 1.73× faster | −4.3% | 6/6 | — | | 2 | registry.npmjs.org | npm | 30.191 | 4.8% | baseline | +9.2% | 6/6 | — | | 3 | JFrog Artifactory | jfrog | 39.463 | 6.4% | 1.31× slower | −4.1% | 6/6 | — | | 4 | GitHub Packages | github | 72.385 | 4.3% | 2.4× slower | +12.3% | 6/6 | — | | 5 | AWS CodeArtifact | aws | 93.38 | 1.9% | 3.1× slower | +8.3% | 6/6 | — | | 6 | Cloudsmith | cloudsmith | 96.87 | 14.1% | 3.2× slower | +10.4% | 6/6 | — | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | | registry.vlt.io | 17.5s | −14.1% | 2026-09-22 | 2026-10-20 | | registry.npmjs.org | 30.2s | −15.6% | 2026-09-22 | 2026-10-20 | | JFrog Artifactory | 39.5s | −19.8% | 2026-09-22 | 2026-10-20 | | GitHub Packages | 72.4s | −13.8% | 2026-09-22 | 2026-10-20 | | AWS CodeArtifact | 93.4s | −16.2% | 2026-09-22 | 2026-10-20 | | Cloudsmith | 96.9s | −0.2% | 2026-09-23 | 2026-10-20 | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/registries/registry-clean/history ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,20.334,35.772,49.196,83.937,111.492, 2026-09-23,17.16,30.353,45.236,71.378,94.329,97.107 2026-09-24,18.261,28.292,43.052,67.318,91.01,113.523 2026-09-25,18.399,35.966,43.374,71.504,94.127,98.974 2026-09-26,20.009,28.002,43.808,62.559,90.493,98.419 2026-09-27,23.005,29.579,46.889,68.694,94.53, 2026-09-28,18.573,28.941,38.658,69.463,91.149, 2026-09-29,17.565,31.182,42.702,78.628,95.051,116.815 2026-09-30,18.429,30.455,37.71,68.89,88.901,83.219 2026-10-01,17.506,29.87,51,75.412,99.101,104.639 2026-10-02,19.446,28.195,50.99,78.087,98.522,112.064 2026-10-03,19.202,28.23,43.23,62.76,91.55,94.772 2026-10-04,18.038,28.25,46.142,65.599,93.113,93.348 2026-10-05,15.974,24.964,44.052,66.849,91.112,96.672 2026-10-06,17.225,26.886,40.908,64.121,84.557,85.257 2026-10-07,18.202,28.451,46.549,72.264,95.731,101.397 2026-10-08,19.002,29.657,43.015,71.015,91.861,96.948 2026-10-09,18.551,27.946,46.738,67.226,90.8,92.982 2026-10-10,16.924,26.219,38.91,60.559,86.251,90.884 2026-10-11,20.276,29.59,43.505,65.901,92.879,94.504 2026-10-12,18.525,27.415,40.205,66.522,88.351,90.946 2026-10-13,17.374,28.52,43.94,70.527,89.534,100.525 2026-10-14,18.798,29.15,41.8,71.473,100.161,95.962 2026-10-15,17.823,29.395,44.331,70.868,93.072,98.088 2026-10-16,17.607,26.852,41.545,71.144,92.71,94.192 2026-10-17,19.045,27.919,46,65.623,97.886,96.576 2026-10-18,16.916,26.884,44.675,67.196,93.728,96.492 2026-10-19,18.264,27.636,41.158,64.436,86.226,87.706 2026-10-20,17.473,30.191,39.463,72.385,93.38,96.87 ``` #### DNF and coverage - Cloudsmith on Babylon.js: DNF on 3 runs (exit 1): 2026-09-22, 2026-09-27, 2026-09-28 - Cloudsmith on Large: DNF on 1 run (exit 1): 2026-09-28 ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 17.5s | 1.73× faster | | registry.npmjs.org | 30.2s | baseline | | JFrog Artifactory | 39.5s | 1.31× slower | | GitHub Packages | 72.4s | 2.4× slower | | AWS CodeArtifact | 93.4s | 3.1× slower | | Cloudsmith | 96.9s | 3.2× slower | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. #### Astro | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 5.83s | 1.81× faster | | registry.npmjs.org | 10.5s | baseline | | JFrog Artifactory | 17.2s | 1.64× slower | | GitHub Packages | 28.1s | 2.7× slower | | AWS CodeArtifact | 29.3s | 2.8× slower | | Cloudsmith | 34.8s | 3.3× slower | #### Babylon.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 52.1s | 1.58× faster | | registry.npmjs.org | 82.1s | baseline | | JFrog Artifactory | 100s | 1.22× slower | | GitHub Packages | 186s | 2.3× slower | | AWS CodeArtifact | 236s | 2.9× slower | | Cloudsmith | 242s | 2.9× slower | #### Large | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 21.5s | 1.86× faster | | registry.npmjs.org | 40.2s | baseline | | JFrog Artifactory | 53.3s | 1.33× slower | | GitHub Packages | 120s | 3.0× slower | | AWS CodeArtifact | 154s | 3.8× slower | | Cloudsmith | 193s | 4.8× slower | #### Next.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 13.0s | 1.63× faster | | registry.npmjs.org | 21.2s | baseline | | JFrog Artifactory | 29.3s | 1.38× slower | | GitHub Packages | 40.4s | 1.90× slower | | Cloudsmith | 54.9s | 2.6× slower | | AWS CodeArtifact | 55.8s | 2.6× slower | #### Svelte | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 2.31s | 2.4× faster | | registry.npmjs.org | 5.60s | baseline | | Cloudsmith | 5.83s | 1.04× slower | | JFrog Artifactory | 6.81s | 1.22× slower | | GitHub Packages | 11.7s | 2.1× slower | | AWS CodeArtifact | 12.9s | 2.3× slower | #### Vue | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 10.0s | 2.1× faster | | registry.npmjs.org | 21.6s | baseline | | JFrog Artifactory | 29.7s | 1.38× slower | | GitHub Packages | 48.2s | 2.2× slower | | Cloudsmith | 51.5s | 2.4× slower | | AWS CodeArtifact | 71.8s | 3.3× slower | ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | | registry.vlt.io | 5.831 | 52.086 | 21.543 | 13.036 | 2.308 | 10.035 | | registry.npmjs.org | 10.528 | 82.062 | 40.167 | 21.218 | 5.602 | 21.569 | | JFrog Artifactory | 17.22 | 100.471 | 53.26 | 29.346 | 6.813 | 29.671 | | GitHub Packages | 28.079 | 185.75 | 120.101 | 40.408 | 11.738 | 48.234 | | AWS CodeArtifact | 29.331 | 236.435 | 154.12 | 55.765 | 12.868 | 71.76 | | Cloudsmith | 34.804 | 241.718 | 192.522 | 54.865 | 5.827 | 51.483 | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 27.6ms | 1.92× faster | | registry.npmjs.org | 53.2ms | baseline | | JFrog Artifactory | 70.9ms | 1.33× slower | | GitHub Packages | 120ms | 2.2× slower | | Cloudsmith | 128ms | 2.4× slower | | AWS CodeArtifact | 149ms | 2.8× slower | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,30.248,58.107,86.107,125.503,166.727,244.152 2026-09-23,26.112,40.232,90.028,118.034,135.428,151.729 2026-09-24,22.613,35.405,46.773,68.837,91.832,128.45 2026-09-25,26.215,50.72,70.071,98.88,113.112,159.476 2026-09-26,25.399,33.981,82.751,103.841,129.203,200.306 2026-09-27,21.033,32.484,43.81,76.44,98.804,158.87 2026-09-28,23.514,37.744,54.661,85.394,102.84,157.049 2026-09-29,20.641,39.434,70.902,111.777,119.552,235.958 2026-09-30,23.863,42.019,50.198,116.548,100.575,110.471 2026-10-01,23.869,36.571,47.902,81.272,93.874,106.516 2026-10-02,24.424,37.045,55.983,90.559,104.432,134.149 2026-10-03,24.004,34.215,61.377,76.093,106.731,137.677 2026-10-04,21.072,33.558,48.151,79.197,98.822,132.534 2026-10-05,21.547,35.081,73.4,99.607,122.343,136.22 2026-10-06,21.369,34.305,50.235,70.436,92.229,100.663 2026-10-07,26.334,38.475,60.269,94.432,106.439,139.623 2026-10-08,22.507,38.494,80.856,103.92,122.182,145.913 2026-10-09,21.369,32.12,45.294,69.015,93.883,147.521 2026-10-10,20.421,31.43,52.832,74.186,98.987,141.973 2026-10-11,20.127,32.155,46.988,79.11,96.778,131.165 2026-10-12,25.617,36.739,63.298,108.588,109.78,139.056 2026-10-13,20.497,32.232,45.869,79.129,90.622,123.474 2026-10-14,33.497,36.032,77.075,106.916,142.046,148.437 2026-10-15,20.778,35.748,74.415,105.421,124.624,151.165 2026-10-16,21.987,33.022,44.069,82.796,109.298,136.985 2026-10-17,21.134,34.624,77.726,96.045,129.786,157.432 2026-10-18,24.719,36.563,65.993,86.874,108.847,129.513 2026-10-19,21.232,32.156,45.293,83.717,105.141,123.937 2026-10-20,20.899,37.736,61.722,100.643,105.128,124.744 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-26: Cloudsmith, +26.8% from its neighbours - 2026-10-06: Cloudsmith, −27.4% from its neighbours - 2026-10-14: registry.vlt.io, +59.8% from its neighbours #### Babylon.js ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,32.176,53.584,70.862,114.343,150.1, 2026-09-23,27.757,44.187,54.855,89.468,117.901,112.773 2026-09-24,27.455,41.945,59.573,89.732,121.811,139.113 2026-09-25,26.996,44.961,56.808,96.031,123.44,131.156 2026-09-26,36.07,45.812,69.19,95.474,132.54,145.36 2026-09-27,46.874,47.655,67.96,94.783,130.7, 2026-09-28,31.958,46.34,57.722,94.462,131.864, 2026-09-29,30.075,50.984,59.714,108.27,131.705,151.436 2026-09-30,29.479,46.767,55.295,97.471,125.341,117.62 2026-10-01,29.489,45.967,65.899,101.458,130.839,135.286 2026-10-02,33.108,43.738,76.49,109.111,138.257,165.381 2026-10-03,33.394,46.194,66.598,91.524,132.889,145.363 2026-10-04,31.261,45.735,65.284,89.872,130.32,130.826 2026-10-05,26.64,39.247,58.63,92.433,123.028,132.394 2026-10-06,28.376,42.755,54.199,87.241,115.202,123.369 2026-10-07,31.53,45.385,61.651,95.149,126.734,121.173 2026-10-08,31.552,45.602,63.768,99.337,130.74,136.556 2026-10-09,31.49,44.418,81.971,107.034,138.359,139.775 2026-10-10,29.261,41.769,56.021,82.893,116.49,122.028 2026-10-11,34.841,48.222,71.119,97.209,134.523,142.472 2026-10-12,30.036,43.237,57.852,91.535,113.535,109.896 2026-10-13,30.554,45.929,60.836,88.707,116.682,128.838 2026-10-14,30.276,49.346,61.752,97.309,138.994,140.679 2026-10-15,31.364,49.561,65.785,109.362,131.912,139.173 2026-10-16,27.025,39.887,53.826,99.054,124.919,128.369 2026-10-17,32.375,44.608,65.14,90.812,135.676,134.301 2026-10-18,27.221,41.482,52.906,85.273,121.823,128.118 2026-10-19,29.576,43.168,57.201,85.819,116.018,121.86 2026-10-20,28.953,45.666,55.848,103.252,131.426,134.138 ``` ##### DNF and coverage - Cloudsmith on Babylon.js: DNF on 3 runs (exit 1): 2026-09-22, 2026-09-27, 2026-09-28 ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-27: registry.vlt.io, +57.4% from its neighbours #### Large ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,18.929,33.109,43.502,89.7,120.999,161.552 2026-09-23,14.313,31.007,56.807,95.528,124.047,153.753 2026-09-24,19.441,27.705,40.421,80.672,106.171,135.776 2026-09-25,22.416,52.135,51.509,92.254,119.271,135.996 2026-09-26,17.294,25.063,36.477,62.109,92.822,117.751 2026-09-27,16.046,28.244,55.282,91.698,118.757,138.827 2026-09-28,15.489,22.727,31.589,75.746,97.755, 2026-09-29,14.383,25.706,36.892,83.177,102.321,142.079 2026-09-30,18.651,27.763,35.873,78.472,100.898,109.795 2026-10-01,14.096,29.153,60.153,95.68,120.888,144.595 2026-10-02,17.671,25.182,62.166,108.651,123.838,147.098 2026-10-03,17.673,26.816,40.996,78.478,105.937,122.004 2026-10-04,16.164,25.976,56.746,90.401,118.449,136.256 2026-10-05,12.772,20.34,46.947,73.734,100.429,126.26 2026-10-06,15.175,23.114,38.995,70.219,88.529,97.689 2026-10-07,15.878,26.084,55.865,95.795,123.288,163.44 2026-10-08,18.584,28.726,37.304,84.115,104.519,129.498 2026-10-09,16.584,23.414,37.558,67.737,91.692,109.914 2026-10-10,14.429,23.093,31.845,71.129,99.493,113.819 2026-10-11,19.513,26.631,40.309,75.634,105.283,119.689 2026-10-12,15.212,22.91,37.285,74.672,104.199,131.546 2026-10-13,14.856,26.747,55.587,98.656,116.624,143.911 2026-10-14,15.562,23.762,32.457,80.706,104.314,111.333 2026-10-15,16.047,25.853,36.992,65.956,93.77,116.142 2026-10-16,18.915,27.645,49.853,87.913,113.011,133.989 2026-10-17,18.04,24.782,42.593,74.961,109.127,119.109 2026-10-18,14.518,24.916,56.263,91.401,119.332,135.814 2026-10-19,17.696,25.704,37.004,74.365,93.967,106.55 2026-10-20,14.816,27.625,36.63,82.601,105.997,132.317 ``` ##### DNF and coverage - Cloudsmith on Large: DNF on 1 run (exit 1): 2026-09-28 ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-25: registry.npmjs.org, +86.4% from its neighbours - 2026-09-27: JFrog Artifactory, +50.7% from its neighbours - 2026-10-07: GitHub Packages, +32.3% from its neighbours - 2026-10-18: JFrog Artifactory, +52.0% from its neighbours #### Next.js ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,38.974,61.956,111.313,185.3,202.981,352.554 2026-09-23,37.139,59.713,87.484,110.395,158.44,156.102 2026-09-24,37.84,53.95,87.804,108.58,155.93,192.836 2026-09-25,33.34,56.479,83.003,107.526,153.698,142.864 2026-09-26,31.444,46.831,75.736,93.32,146.372,128.833 2026-09-27,40.435,57.952,91.378,110.225,157.782,174.448 2026-09-28,35.16,50.62,76.294,92.401,135.117,141.415 2026-09-29,33.58,56.471,72.891,133.796,155.202,175.484 2026-09-30,34.973,55.105,76.749,105.786,146.723,127.007 2026-10-01,42.605,56.712,100.483,124.397,176.913,165.594 2026-10-02,35.211,53.222,75.141,90.085,138.028,121.835 2026-10-03,32.841,48.055,78.225,85.15,142.4,119.672 2026-10-04,33.351,50.205,81.402,86.105,138.811,114.99 2026-10-05,36.807,50.958,107.669,136.073,176.836,161.213 2026-10-06,34.989,53.889,76.251,111.659,151.64,133.03 2026-10-07,31.419,51.258,106.17,118.679,169.001,155.163 2026-10-08,34.68,53.549,86.747,112.204,146.57,134.518 2026-10-09,41.432,63.37,90.765,110.678,161.251,139.737 2026-10-10,32.993,46.776,78.955,87.082,135.708,116.671 2026-10-11,37.04,54.401,81.309,101.861,166.362,128.694 2026-10-12,44.555,48.505,75.764,105.835,151.779,141.845 2026-10-13,31.723,50.178,76.103,114.623,148.511,170.609 2026-10-14,46.044,53.648,109.064,140.998,179.779,149.975 2026-10-15,31.208,47.693,70.259,113.654,152.676,125.136 2026-10-16,37.196,53.015,94.687,111.06,162.686,146.455 2026-10-17,36.546,53.663,88.882,105.932,160.734,144.198 2026-10-18,37.531,51.223,101.505,121.9,176.413,165.832 2026-10-19,36.151,53.461,110.419,129.14,177.902,177.896 2026-10-20,38.912,63.338,87.599,120.621,166.463,163.289 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-10-01: JFrog Artifactory, +31.3% from its neighbours - 2026-10-01: AWS CodeArtifact, +25.8% from its neighbours - 2026-10-05: JFrog Artifactory, +34.9% from its neighbours - 2026-10-14: GitHub Packages, +30.0% from its neighbours #### Svelte ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,51.748,138.741,154.972,248.81,349.635,230.064 2026-09-23,44.333,108.121,127.778,188.92,238.723,127.465 2026-09-24,37.874,89.769,110.41,155.137,200.855,125.566 2026-09-25,45.045,107,123.93,183.04,236.709,146.359 2026-09-26,41.253,91.742,116.133,150.46,229.768,134.741 2026-09-27,35.588,80.806,108.12,150.921,215.416,120.477 2026-09-28,44.617,105.902,168.372,184.089,251.933,127.229 2026-09-29,40.991,122.126,165.704,223.127,272.649,129.143 2026-09-30,35.436,92.478,117.335,175.907,237.253,109.7 2026-10-01,35.982,93.934,203.169,201.401,266.999,186.515 2026-10-02,37.208,88.274,166.579,204.128,255.533,134.098 2026-10-03,35.783,75.574,119.686,142.016,204.482,107.896 2026-10-04,41.512,90.646,131.937,166.523,234.938,110.756 2026-10-05,41.886,88.075,124.666,161.815,277.598,102.548 2026-10-06,41.92,88.713,115.85,151.463,215.938,131.688 2026-10-07,41.269,85.311,119.035,150.215,210.392,96.611 2026-10-08,40.327,88.179,116.103,144.477,210.502,113.965 2026-10-09,43.324,100.54,134.565,172.658,240.272,134.305 2026-10-10,37.831,81.602,110.223,156.069,217.262,105.954 2026-10-11,39.256,83.476,113.607,145.267,204.863,92.293 2026-10-12,36.048,85.288,106.3,169.957,204.39,127.666 2026-10-13,43.741,94.06,143.608,187.834,228.088,172.899 2026-10-14,37.804,91.817,170.376,209.378,262.299,127.672 2026-10-15,42.915,86.887,140.027,197.902,255.621,149.473 2026-10-16,36.856,85.041,114.504,156.516,204.361,98.086 2026-10-17,44.097,88.468,123.275,164.179,244.482,128.438 2026-10-18,38.219,88.175,166.974,194.799,257.58,151.121 2026-10-19,42.894,91.14,172.309,198.299,287.276,126.34 2026-10-20,39.796,96.589,117.472,202.382,221.865,100.467 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-29: registry.npmjs.org, +32.6% from its neighbours - 2026-10-01: Cloudsmith, +56.7% from its neighbours - 2026-10-05: AWS CodeArtifact, +30.2% from its neighbours - 2026-10-19: JFrog Artifactory, +43.1% from its neighbours #### Vue ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,27.051,56.357,76.425,125.273,197.452,174.216 2026-09-23,22.296,45.226,63.657,106.873,154.566,119.583 2026-09-24,24.094,47.024,97.637,136.036,186.404,278.66 2026-09-25,18.307,44.773,64.297,107.628,162.797,130.962 2026-09-26,22.495,42.136,67.767,99.628,159.809,112.38 2026-09-27,19.938,39.488,66.34,92.639,149.018,148.659 2026-09-28,23.543,53.131,70.857,159.76,177.056,254.3 2026-09-29,24.629,47.305,92.68,150.446,187.069,198.202 2026-09-30,22.403,51.073,63.006,102.543,157.481,106.351 2026-10-01,18.992,43.902,92.315,122.876,185.013,172.582 2026-10-02,23.881,46.351,61.642,105.312,161.777,165.257 2026-10-03,21.675,38.907,66.574,89.259,149.906,98.65 2026-10-04,20.238,40.595,63.987,89.396,144.195,98.101 2026-10-05,19.227,39.717,60.089,100.532,156.601,132.803 2026-10-06,22.7,40.606,100.827,134.16,184.566,148.492 2026-10-07,20.488,41.185,62.488,108.311,153.85,123.677 2026-10-08,22.403,45.82,69.01,112.37,153.805,121.778 2026-10-09,18.356,39.889,61.664,103.065,153.42,111.792 2026-10-10,20.136,43.739,86.979,116.338,173.983,166.692 2026-10-11,23.111,45.13,61.748,102.986,151.504,113.478 2026-10-12,24.288,49.405,75.634,112.739,181.659,138.933 2026-10-13,19.713,41.342,59.981,109.334,155.738,133.556 2026-10-14,19.469,40.882,54.884,105.425,187.994,135.399 2026-10-15,18.81,41.789,92.55,119.671,187.712,168.268 2026-10-16,19.601,38.69,65.419,114.892,157.137,101.405 2026-10-17,20.289,41.409,85.475,110.689,179.405,142.633 2026-10-18,21.701,40.306,64.675,90.415,145.797,110.499 2026-10-19,22.587,41.741,68.369,102.674,148.321,110.689 2026-10-20,22.449,48.254,66.378,107.906,160.537,114.407 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-24: JFrog Artifactory, +47.2% from its neighbours - 2026-09-24: Cloudsmith, +112.8% from its neighbours - 2026-09-28: GitHub Packages, +52.0% from its neighbours - 2026-09-29: JFrog Artifactory, +38.2% from its neighbours - 2026-09-29: GitHub Packages, +44.8% from its neighbours - 2026-10-01: JFrog Artifactory, +41.4% from its neighbours - 2026-10-06: JFrog Artifactory, +59.4% from its neighbours - 2026-10-10: JFrog Artifactory, +40.0% from its neighbours - 2026-10-17: JFrog Artifactory, +29.7% from its neighbours ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | | registry.vlt.io | 20.899 | 28.953 | 14.816 | 38.912 | 39.796 | 22.449 | | registry.npmjs.org | 37.736 | 45.666 | 27.625 | 63.338 | 96.589 | 48.254 | | JFrog Artifactory | 61.722 | 55.848 | 36.63 | 87.599 | 117.472 | 66.378 | | GitHub Packages | 100.643 | 103.252 | 82.601 | 120.621 | 202.382 | 107.906 | | AWS CodeArtifact | 105.128 | 131.426 | 105.997 | 166.463 | 221.865 | 160.537 | | Cloudsmith | 124.744 | 134.138 | 132.317 | 163.289 | 100.467 | 114.407 | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | | registry.npmjs.org | 279 | 1797 (1791–1801) | 1454 | 335 | 58 | 447 | 4370 | | registry.vlt.io | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | AWS CodeArtifact | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | GitHub Packages | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | JFrog Artifactory | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | Cloudsmith | 279 | 1802 | 1455 | 336 | 58 | 450 | 4380 | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. - Disclosure. vlt publishes this benchmark, and registry.vlt.io is one of the registries it measures. The suite, its configuration and every result file are public. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - Raw results, Astro (JSON): https://benchmarks-data.vlt.io/2026-10-20/astro-registry-clean.json - Raw results, Babylon.js (JSON): https://benchmarks-data.vlt.io/2026-10-20/babylon-registry-clean.json - Raw results, Large (JSON): https://benchmarks-data.vlt.io/2026-10-20/large-registry-clean.json - Raw results, Next.js (JSON): https://benchmarks-data.vlt.io/2026-10-20/next-registry-clean.json - Raw results, Svelte (JSON): https://benchmarks-data.vlt.io/2026-10-20/svelte-registry-clean.json - Raw results, Vue (JSON): https://benchmarks-data.vlt.io/2026-10-20/vue-registry-clean.json - This report as JSON: https://benchmarks.vlt.io/api/v1/registries/registry-clean - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/registries/registry-clean?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Registries, registry-clean [Data set]. https://benchmarks.vlt.io/registries/registry-clean?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Registries, registry-clean}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/registries/registry-clean?date=2026-10-20} } ``` # Registries — registry-lockfile (https://benchmarks.vlt.io/registries/registry-lockfile) Benchmarks / Registries / registry-lockfile - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. Every registry is installed from with the same npm client. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline registry.vlt.io is fastest at 10.8s, 1.03× faster than registry.npmjs.org. - Fastest mean install: 10.8s (registry.vlt.io) - Against registry.npmjs.org: 1.03× (faster, on the mean) - Fixtures won: 4 / 6 (where registry.vlt.io was fastest outright) - Spread: 6.0× (between the slowest and fastest tools that completed every fixture) ## Scenario A lockfile install, holding the package manager constant and varying the registry. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against registry.npmjs.org | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | | 1 | registry.vlt.io | vlt | 10.826 | 2.4% | 1.03× faster | −0.6% | 6/6 | — | | 2 | registry.npmjs.org | npm | 11.137 | 0.6% | baseline | +0.6% | 6/6 | — | | 3 | JFrog Artifactory | jfrog | 12.482 | 1.0% | 1.12× slower | +2.7% | 6/6 | — | | 4 | GitHub Packages | github | 12.991 | 1.5% | 1.17× slower | +2.4% | 6/6 | — | | 5 | AWS CodeArtifact | aws | 15.337 | 1.5% | 1.38× slower | +11.2% | 6/6 | — | | 6 | Cloudsmith | cloudsmith | 64.793 | 49.4% | 5.8× slower | +16.5% | 6/6 | — | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | | registry.vlt.io | 10.8s | +1.7% | 2026-09-23 | 2026-10-20 | | registry.npmjs.org | 11.1s | +2.2% | 2026-09-22 | 2026-10-20 | | JFrog Artifactory | 12.5s | +1.0% | 2026-09-22 | 2026-10-20 | | GitHub Packages | 13.0s | +0.6% | 2026-09-22 | 2026-10-20 | | AWS CodeArtifact | 15.3s | −0.2% | 2026-09-22 | 2026-10-20 | | Cloudsmith | 64.8s | +444.2% | 2026-09-23 | 2026-10-20 | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/registries/registry-lockfile/history ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,,10.893,12.361,12.912,15.374, 2026-09-23,10.649,10.837,11.746,12.488,23.983,11.905 2026-09-24,10.791,10.932,11.932,12.44,13.908,23.92 2026-09-25,10.865,11.001,12.394,13.444,14.944,51.023 2026-09-26,10.455,10.61,11.777,12.069,13.587,14.253 2026-09-27,10.607,10.799,12.242,12.648,13.75,55.865 2026-09-28,11.117,11.255,12.146,12.639,13.557, 2026-09-29,10.934,10.978,12.116,13.029,14.211, 2026-09-30,10.756,10.943,11.844,12.524,13.533,11.968 2026-10-01,11.002,10.995,12.116,12.634,14.036,24.472 2026-10-02,10.67,10.793,11.902,12.419,13.721,24.611 2026-10-03,10.604,10.709,11.698,12.223,13.716,25.777 2026-10-04,10.44,10.86,11.779,12.378,14.099,33.147 2026-10-05,11.12,11.303,12.503,13.426,15.296,12.839 2026-10-06,10.725,10.739,11.855,12.338,13.721,12.095 2026-10-07,10.815,10.911,12.405,13.028,14.875,45.728 2026-10-08,10.458,10.665,11.713,12.318,13.877,55.362 2026-10-09,10.737,10.736,12.131,12.442,13.988, 2026-10-10,10.633,10.775,11.813,12.388,13.422,26.449 2026-10-11,10.749,10.842,11.788,12.235,13.254,12.225 2026-10-12,11.358,11.581,12.493,13.104,14.978,12.732 2026-10-13,10.861,11.577,12.284,13.277,14.68,12.661 2026-10-14,11.445,11.313,12.652,13.246,15.159,23.557 2026-10-15,10.597,10.718,11.665,12.404,14.22,30.025 2026-10-16,10.74,10.853,12.112,12.482,14.285,25.065 2026-10-17,11.15,11.424,12.446,12.865,14.471,30.565 2026-10-18,10.953,11.266,12.001,12.526,13.769,23.64 2026-10-19,10.887,11.066,12.153,12.686,13.791,55.613 2026-10-20,10.826,11.137,12.482,12.991,15.337,64.793 ``` #### DNF and coverage - registry.vlt.io on Large: DNF on 1 run (exit 1): 2026-09-22 - Cloudsmith on Large: DNF on 1 run (exit 1): 2026-09-22 - Cloudsmith on Babylon.js: DNF on 3 runs (exit 1): 2026-09-28, 2026-09-29, 2026-10-09 #### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-23: AWS CodeArtifact, +66.2% from its neighbours - 2026-09-25: Cloudsmith, +258.0% from its neighbours - 2026-09-27: Cloudsmith, +130.9% from its neighbours - 2026-10-08: Cloudsmith, +333.0% from its neighbours - 2026-10-10: Cloudsmith, +108.3% from its neighbours - 2026-10-19: Cloudsmith, +99.9% from its neighbours ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 10.8s | 1.03× faster | | registry.npmjs.org | 11.1s | baseline | | JFrog Artifactory | 12.5s | 1.12× slower | | GitHub Packages | 13.0s | 1.17× slower | | AWS CodeArtifact | 15.3s | 1.38× slower | | Cloudsmith | 64.8s | 5.8× slower | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. #### Astro | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | AWS CodeArtifact | 4.05s | 1.02× faster | | registry.vlt.io | 4.05s | 1.02× faster | | registry.npmjs.org | 4.12s | baseline | | GitHub Packages | 4.39s | 1.06× slower | | Cloudsmith | 4.47s | 1.08× slower | | JFrog Artifactory | 4.50s | 1.09× slower | #### Babylon.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.npmjs.org | 34.9s | baseline | | registry.vlt.io | 35.2s | 1.01× slower | | GitHub Packages | 36.6s | 1.05× slower | | AWS CodeArtifact | 36.9s | 1.06× slower | | JFrog Artifactory | 37.1s | 1.06× slower | | Cloudsmith | 298s | 8.5× slower | #### Large | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 13.2s | 1.13× faster | | registry.npmjs.org | 15.0s | baseline | | JFrog Artifactory | 19.6s | 1.31× slower | | GitHub Packages | 23.4s | 1.56× slower | | AWS CodeArtifact | 37.5s | 2.5× slower | | Cloudsmith | 72.6s | 4.8× slower | #### Next.js | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 7.28s | 1.01× faster | | registry.npmjs.org | 7.36s | baseline | | AWS CodeArtifact | 7.42s | 1.01× slower | | JFrog Artifactory | 7.46s | 1.01× slower | | Cloudsmith | 7.51s | 1.02× slower | | GitHub Packages | 7.54s | 1.02× slower | #### Svelte | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 692ms | 1.09× faster | | registry.npmjs.org | 752ms | baseline | | Cloudsmith | 766ms | 1.02× slower | | GitHub Packages | 924ms | 1.23× slower | | JFrog Artifactory | 1.01s | 1.35× slower | | AWS CodeArtifact | 1.21s | 1.61× slower | #### Vue | Tool | Time | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 4.49s | 1.04× faster | | registry.npmjs.org | 4.67s | baseline | | AWS CodeArtifact | 4.87s | 1.04× slower | | Cloudsmith | 5.03s | 1.08× slower | | GitHub Packages | 5.07s | 1.09× slower | | JFrog Artifactory | 5.21s | 1.12× slower | ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | | registry.vlt.io | 4.052 | 35.205 | 13.239 | 7.282 | 0.692 | 4.487 | | registry.npmjs.org | 4.121 | 34.933 | 14.991 | 7.359 | 0.752 | 4.665 | | JFrog Artifactory | 4.502 | 37.103 | 19.605 | 7.457 | 1.013 | 5.21 | | GitHub Packages | 4.388 | 36.613 | 23.409 | 7.538 | 0.924 | 5.075 | | AWS CodeArtifact | 4.05 | 36.932 | 37.539 | 7.418 | 1.214 | 4.869 | | Cloudsmith | 4.465 | 298.425 | 72.563 | 7.511 | 0.766 | 5.029 | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against registry.npmjs.org | | --- | --- | --- | | registry.vlt.io | 14.5ms | 1.04× faster | | registry.npmjs.org | 15.0ms | baseline | | JFrog Artifactory | 16.9ms | 1.13× slower | | GitHub Packages | 17.0ms | 1.13× slower | | AWS CodeArtifact | 19.1ms | 1.28× slower | | Cloudsmith | 46.4ms | 3.1× slower | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,13.887,13.746,15.483,15.427,18.139,14.946 2026-09-23,13.455,13.464,14.858,14.679,13.512,14.719 2026-09-24,13.59,13.585,15.508,15.102,17.12,14.922 2026-09-25,13.439,14.108,15.414,15.098,13.942,14.983 2026-09-26,12.402,12.574,13.913,13.859,13.715,14.365 2026-09-27,13.611,13.634,15.442,15.081,14.04,15.265 2026-09-28,13.303,13.294,14.673,14.702,13.294,14.919 2026-09-29,13.628,13.76,15.664,15.083,17.818,14.951 2026-09-30,13.524,13.348,14.728,14.682,13.099,14.175 2026-10-01,13.145,13.44,14.64,14.139,13.229,14.746 2026-10-02,13.557,13.638,14.438,14.435,13.856,14.427 2026-10-03,14.029,14.192,15.428,15.587,14.385,15.544 2026-10-04,13.42,14.113,15.883,15.387,17.07,15.074 2026-10-05,12.781,13.106,14.056,14.151,15.035,14.363 2026-10-06,13.578,13.975,15.083,14.95,14.527,15.048 2026-10-07,12.688,12.458,14.7,14.077,17.29,13.794 2026-10-08,12.671,13.297,13.976,14.256,13.943,13.804 2026-10-09,13.169,13.684,15.298,15.014,13.456,14.827 2026-10-10,13.175,13.789,15.262,15.177,15.49,14.021 2026-10-11,13.84,13.623,14.927,14.634,13.52,14.965 2026-10-12,14.051,14.278,15.384,15.255,14.388,15.35 2026-10-13,12.482,12.562,14.245,14.041,14.125,17.553 2026-10-14,13.053,13.268,15.002,15.067,14.277,15.642 2026-10-15,12.953,12.994,14.052,14.084,13.72,13.893 2026-10-16,13.248,13.753,15.323,15.185,14.825,16.675 2026-10-17,12.506,12.965,14.177,14.137,14.519,14.004 2026-10-18,13.331,13.56,15.084,14.507,14.21,14.781 2026-10-19,13.605,13.236,15.31,14.423,17.109,15.027 2026-10-20,14.523,14.771,16.137,15.728,14.517,16.005 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-29: AWS CodeArtifact, +31.9% from its neighbours #### Babylon.js ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,19.022,18.973,20.757,20.207,19.697,65.433 2026-09-23,19.287,18.885,20.27,19.805,52.693,20.339 2026-09-24,19.402,19.215,20.077,20.268,19.268,59.893 2026-09-25,18.759,18.688,20.259,19.726,18.687,74.196 2026-09-26,18.498,18.032,19.567,19.017,18.408,19.697 2026-09-27,18.572,18.465,20.732,20.308,18.87,165.29 2026-09-28,19.944,19.664,20.896,20.475,19.448, 2026-09-29,19.621,19.121,20.615,20.509,20.477, 2026-09-30,19.081,18.738,19.945,19.783,18.684,20.192 2026-10-01,20.009,19.387,21.06,20.364,19.454,60.074 2026-10-02,18.974,18.648,20.002,19.656,18.842,62.118 2026-10-03,18.792,18.486,19.712,19.252,18.816,65.679 2026-10-04,18.226,18.59,19.49,19.192,19.789,90.314 2026-10-05,20.024,19.76,21.226,21.274,20.113,21.706 2026-10-06,19.217,18.42,19.92,19.644,19.036,20.476 2026-10-07,18.884,18.578,20.124,19.608,19.159,130.87 2026-10-08,18.388,18.325,19.644,19.24,19.01,165.154 2026-10-09,18.832,17.913,19.317,19.198,18.795, 2026-10-10,19.192,18.957,20.468,20.42,19.257,68.631 2026-10-11,19.043,18.573,19.706,19.414,18.707,20.195 2026-10-12,20.858,20.736,22.151,21.762,20.924,22.732 2026-10-13,19.533,19.519,20.541,19.992,19.537,20.692 2026-10-14,20.277,19.604,20.924,20.706,19.883,57.179 2026-10-15,18.683,18.526,19.594,19.428,18.972,80.045 2026-10-16,19.105,18.916,20.362,19.765,19.807,63.192 2026-10-17,20.644,20.483,21.788,21.537,20.504,81.968 2026-10-18,19.908,20.164,20.707,20.469,19.905,58.565 2026-10-19,18.981,18.845,20.495,20.143,19.254,164.623 2026-10-20,19.547,19.505,20.624,20.352,20.541,165.608 ``` ##### DNF and coverage - Cloudsmith on Babylon.js: DNF on 3 runs (exit 1): 2026-09-28, 2026-09-29, 2026-10-09 ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-23: AWS CodeArtifact, +177.7% from its neighbours - 2026-09-27: Cloudsmith, +175.6% from its neighbours - 2026-09-30: Cloudsmith, −68.4% from its neighbours - 2026-10-08: Cloudsmith, +643.3% from its neighbours - 2026-10-10: Cloudsmith, +216.1% from its neighbours - 2026-10-19: Cloudsmith, +126.8% from its neighbours #### Large ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,,9.842,12.037,15.362,25.257, 2026-09-23,8.736,9.92,10.774,14.493,22.007,11.417 2026-09-24,9.481,10.248,11.567,14.05,20.09,12.368 2026-09-25,9.726,10.373,12.992,18.093,25.97,105.339 2026-09-26,9.026,10.032,11.871,13.828,21.342,21.982 2026-09-27,9.029,9.842,11.491,13.807,20.222,12.58 2026-09-28,9.766,10.668,11.59,14.102,19.776,20.272 2026-09-29,9.228,10.038,11.554,14.447,20.343,12.337 2026-09-30,9.126,10.388,11.248,14.353,20.832,11.624 2026-10-01,8.915,9.46,10.748,13.865,21.658,13.446 2026-10-02,8.966,9.907,11.664,14.327,21.105,12.078 2026-10-03,8.95,9.704,11.242,13.907,21.462,12.34 2026-10-04,8.812,9.82,11.359,14.302,20.839,11.843 2026-10-05,9.758,10.452,12.851,16.572,25.745,12.849 2026-10-06,8.998,9.979,11.389,14.202,20.943,11.967 2026-10-07,9.652,10.544,13.026,16.911,24.683,14.067 2026-10-08,8.971,9.766,11.301,14.349,20.927,11.328 2026-10-09,9.212,10.135,12.485,14.576,21.499,13.959 2026-10-10,8.545,9.294,10.586,13.067,18.989,11.356 2026-10-11,9.135,10.145,11.383,13.739,19.609,12.705 2026-10-12,9.364,10.552,11.375,14.416,23.642,11.811 2026-10-13,9.543,10.447,12.829,16.022,23.962,12.8 2026-10-14,10.05,10.492,13.208,16.009,25.28,13.136 2026-10-15,8.928,9.816,11.085,14.436,22.781,11.97 2026-10-16,8.924,9.727,11.064,14.18,21.296,12.002 2026-10-17,9.123,10.402,11.478,13.939,21.687,11.931 2026-10-18,8.943,9.942,11.292,13.738,20.314,12.075 2026-10-19,9.692,10.589,11.852,14.831,20.31,12.704 2026-10-20,9.105,10.31,13.483,16.1,25.818,49.872 ``` ##### DNF and coverage - registry.vlt.io on Large: DNF on 1 run (exit 1): 2026-09-22 - Cloudsmith on Large: DNF on 1 run (exit 1): 2026-09-22 ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-25: GitHub Packages, +28.5% from its neighbours - 2026-09-25: Cloudsmith, +737.3% from its neighbours - 2026-09-26: Cloudsmith, +76.2% from its neighbours - 2026-09-28: Cloudsmith, +55.8% from its neighbours #### Next.js ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,21.438,21.641,22.784,23.044,22.428,23.404 2026-09-23,20.726,21.184,22.77,22.522,22.002,22.557 2026-09-24,20.331,20.304,22.312,21.6,23.366,21.387 2026-09-25,22.689,22.682,24.084,24.121,23.97,24.313 2026-09-26,20.774,21.599,22.524,22.49,21.722,22.274 2026-09-27,21.966,22.019,23.865,23.989,23.055,23.894 2026-09-28,21.308,21.505,22.934,23.007,23.343,23.185 2026-09-29,21.629,21.533,22.339,23.124,22.158,22.596 2026-09-30,21.006,21.041,23.015,23.099,22.295,23.074 2026-10-01,22.41,22.653,23.922,24.356,22.886,24.622 2026-10-02,21.357,21.434,22.533,22.742,22.587,22.64 2026-10-03,21.554,21.781,22.571,23.051,21.635,23.148 2026-10-04,21.452,22.046,22.905,23.588,22.912,23.093 2026-10-05,21.129,21.898,22.272,22.402,21.474,22.422 2026-10-06,20.334,20.755,21.98,22.002,22.536,21.924 2026-10-07,21.945,22.208,23.582,23.409,22.491,23.36 2026-10-08,21.609,22.222,23.174,23.126,25.375,23.007 2026-10-09,22.587,23.387,24.529,24.524,24.569,24.355 2026-10-10,22.202,22.396,23.214,23.198,23.329,23.667 2026-10-11,21.381,21.781,22.897,22.367,22.396,22.804 2026-10-12,21.094,21.001,22.13,22.145,21.06,22.183 2026-10-13,20.457,20.835,21.832,21.836,21.569,22.056 2026-10-14,23.966,23.336,24.126,24.133,24.905,24.378 2026-10-15,22.452,22.005,23.783,23.379,23.352,23.564 2026-10-16,22.071,22.077,26.953,23.44,25.478,23.655 2026-10-17,20.985,21.221,22.08,22.494,22.15,22.653 2026-10-18,21.27,21.307,22.22,22.201,21.396,22.853 2026-10-19,21.71,21.943,22.961,22.676,22.091,22.564 2026-10-20,21.738,21.966,22.259,22.501,22.144,22.355 ``` #### Svelte ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,22.933,23.527,36.797,26.747,50.898,24.965 2026-09-23,21.905,23.32,25.606,27.291,24.479,24.346 2026-09-24,21.691,23.569,33.886,25.779,38.173,24.586 2026-09-25,22.976,23.787,26.633,26.294,26.084,25.051 2026-09-26,21.687,22.981,24.982,25.836,27.388,23.519 2026-09-27,22.349,24.146,27.399,27.623,30.57,25.01 2026-09-28,22.92,24.848,27.886,28.702,27.728,24.563 2026-09-29,21.327,22.125,25.981,55.219,29.11,23.425 2026-09-30,23.465,25.341,26.17,25.704,23.653,24.561 2026-10-01,23.358,27.643,30.785,31.122,28.881,25.366 2026-10-02,23.096,25.131,27.78,28.909,29.987,25.709 2026-10-03,21.624,22.758,25.12,26.011,25.581,23.251 2026-10-04,21.242,22.016,24.522,24.5,24.202,23.271 2026-10-05,22.429,24.359,26.034,26.608,30.389,24.555 2026-10-06,21.183,21.964,24.771,24.258,22.522,22.872 2026-10-07,22.121,22.977,33.459,25.718,35.278,23.517 2026-10-08,22.267,23.713,26.411,27.342,26.885,24.806 2026-10-09,22.917,24.104,39.079,26.705,38.603,24.514 2026-10-10,22.804,24.921,27.313,28.945,27.21,24.19 2026-10-11,22.297,23.951,27.719,28.725,26.817,24.029 2026-10-12,22.831,24.721,26.698,27.143,27.509,25.441 2026-10-13,23.09,24.512,26.444,26.576,25.141,29.919 2026-10-14,21.883,22.972,25.028,25.521,24.426,23.609 2026-10-15,22.59,24.453,27.133,28.787,25.097,27.078 2026-10-16,22.658,24.942,26.821,29.099,26.436,24.153 2026-10-17,22.596,23.74,35.832,26.608,37.115,24.36 2026-10-18,23.116,23.539,25.731,26.277,23.769,24.468 2026-10-19,21.845,23.903,26.726,26.192,24.636,24.557 2026-10-20,11.929,12.96,17.461,15.929,20.922,13.199 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-09-24: JFrog Artifactory, +27.2% from its neighbours - 2026-09-29: GitHub Packages, +96.1% from its neighbours - 2026-10-09: JFrog Artifactory, +44.7% from its neighbours - 2026-10-09: AWS CodeArtifact, +42.7% from its neighbours - 2026-10-17: JFrog Artifactory, +36.6% from its neighbours - 2026-10-17: AWS CodeArtifact, +51.3% from its neighbours #### Vue ```csv date,vlt,npm,jfrog,github,aws,cloudsmith 2026-09-22,10.283,10.299,11.683,11.617,10.227,11.368 2026-09-23,10.093,10.275,11.444,11.248,10.334,11.329 2026-09-24,9.345,9.717,10.997,10.812,10.747,10.77 2026-09-25,10.301,10.216,11.505,11.358,10.962,11.992 2026-09-26,10.374,10.576,11.977,11.691,10.571,11.577 2026-09-27,10.347,10.843,12.455,12.173,12.902,11.798 2026-09-28,9.877,9.978,11.3,11.253,10.055,11.267 2026-09-29,10.233,10.515,12.223,11.463,10.73,11.929 2026-09-30,10.608,10.656,12.319,12.014,10.81,11.98 2026-10-01,10.049,10.203,11.878,11.439,10.574,11.488 2026-10-02,10.172,10.112,11.846,11.209,10.294,11.209 2026-10-03,9.819,10.006,11.345,10.993,10.125,10.894 2026-10-04,10.855,11.183,12.483,11.948,10.927,12.448 2026-10-05,10.139,10.823,11.78,11.648,11.268,13.934 2026-10-06,10.825,10.788,12.835,11.419,10.624,11.95 2026-10-07,10.472,10.344,11.991,11.309,10.107,11.154 2026-10-08,10.139,9.964,11.905,11.478,10.54,11.281 2026-10-09,10.187,10.192,11.499,11.136,10.435,11.557 2026-10-10,9.808,9.841,11.315,11.006,10.273,11.644 2026-10-11,10.31,10.204,11.844,11.81,10.197,11.672 2026-10-12,10.443,10.206,11.922,11.797,11.672,11.214 2026-10-13,9.953,16.611,11.824,17.102,12.307,13.491 2026-10-14,10.244,10.383,11.909,11.546,10.481,11.573 2026-10-15,10.046,9.886,11.507,11.254,11.104,11.211 2026-10-16,10.365,9.845,11.347,11.007,10.98,11.087 2026-10-17,10.308,10.309,11.942,11.482,10.697,11.801 2026-10-18,10.415,10.554,11.563,11.925,10.647,12.125 2026-10-19,10.49,10.733,11.813,11.527,11.121,11.519 2026-10-20,10.038,10.436,11.656,11.352,10.893,11.175 ``` ##### Flagged outliers Runs far from the median of their neighbours (more than 4 robust standard deviations and 25%). - 2026-10-13: registry.npmjs.org, +65.4% from its neighbours - 2026-10-13: GitHub Packages, +50.0% from its neighbours ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | | registry.vlt.io | 14.523 | 19.547 | 9.105 | 21.738 | 11.929 | 10.038 | | registry.npmjs.org | 14.771 | 19.505 | 10.31 | 21.966 | 12.96 | 10.436 | | JFrog Artifactory | 16.137 | 20.624 | 13.483 | 22.259 | 17.461 | 11.656 | | GitHub Packages | 15.728 | 20.352 | 16.1 | 22.501 | 15.929 | 11.352 | | AWS CodeArtifact | 14.517 | 20.541 | 25.818 | 22.144 | 20.922 | 10.893 | | Cloudsmith | 16.005 | 165.608 | 49.872 | 22.355 | 13.199 | 11.175 | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | | registry.npmjs.org | 279 | 1791 | 1454 | 335 | 58 | 447 | 4364 | | AWS CodeArtifact | 279 | 1798 | 1454 | 335 | 58 | 447 | 4371 | | GitHub Packages | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | JFrog Artifactory | 279 | 1799 | 1454 | 335 | 58 | 447 | 4372 | | registry.vlt.io | 279 | 1801 | 1454 | 335 | 58 | 447 | 4374 | | Cloudsmith | 279 | 1802 | 1455 | 336 | 58 | 450 | 4380 | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. - Disclosure. vlt publishes this benchmark, and registry.vlt.io is one of the registries it measures. The suite, its configuration and every result file are public. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - Raw results, Astro (JSON): https://benchmarks-data.vlt.io/2026-10-20/astro-registry-lockfile.json - Raw results, Babylon.js (JSON): https://benchmarks-data.vlt.io/2026-10-20/babylon-registry-lockfile.json - Raw results, Large (JSON): https://benchmarks-data.vlt.io/2026-10-20/large-registry-lockfile.json - Raw results, Next.js (JSON): https://benchmarks-data.vlt.io/2026-10-20/next-registry-lockfile.json - Raw results, Svelte (JSON): https://benchmarks-data.vlt.io/2026-10-20/svelte-registry-lockfile.json - Raw results, Vue (JSON): https://benchmarks-data.vlt.io/2026-10-20/vue-registry-lockfile.json - This report as JSON: https://benchmarks.vlt.io/api/v1/registries/registry-lockfile - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/registries/registry-lockfile?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Registries, registry-lockfile [Data set]. https://benchmarks.vlt.io/registries/registry-lockfile?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Registries, registry-lockfile}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/registries/registry-lockfile?date=2026-10-20} } ``` # Package managers — average (https://benchmarks.vlt.io/package-managers/average) Benchmarks / Package managers / average - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, averaged over 9 scenarios (derived, not measured), on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Every tool across all of this benchmark's scenarios at once. Mean of: clean, cache, lockfile, node_modules, cache+lockfile, cache+node_modules, lockfile+node_modules, cache+lockfile+node_modules, ci. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. Each run here averages 54 result files, so the trend covers the last 7 runs of the window. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/average/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/average - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/average?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, average [Data set]. https://benchmarks.vlt.io/package-managers/average?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, average}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/average?date=2026-10-20} } ``` # Package managers — clean (https://benchmarks.vlt.io/package-managers/clean) Benchmarks / Package managers / clean - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario A first install on a fresh machine. Every package is resolved, downloaded and linked from scratch. Before each run: binary preinstalled, cache empty, lockfile absent, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/clean/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/clean - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/clean?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, clean [Data set]. https://benchmarks.vlt.io/package-managers/clean?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, clean}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/clean?date=2026-10-20} } ``` # Package managers — cache (https://benchmarks.vlt.io/package-managers/cache) Benchmarks / Package managers / cache - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario The package cache is warm. Nothing else is: no lockfile, no node_modules. Before each run: binary preinstalled, cache present, lockfile absent, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/cache/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/cache - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/cache?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, cache [Data set]. https://benchmarks.vlt.io/package-managers/cache?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, cache}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/cache?date=2026-10-20} } ``` # Package managers — lockfile (https://benchmarks.vlt.io/package-managers/lockfile) Benchmarks / Package managers / lockfile - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario A lockfile pins the graph, but nothing is cached and nothing is installed. Resolution is skipped; download is not. Before each run: binary preinstalled, cache empty, lockfile present, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/lockfile/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/lockfile - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/lockfile?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, lockfile [Data set]. https://benchmarks.vlt.io/package-managers/lockfile?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, lockfile}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/lockfile?date=2026-10-20} } ``` # Package managers — node_modules (https://benchmarks.vlt.io/package-managers/node_modules) Benchmarks / Package managers / node_modules - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario node_modules is already populated. The question is how quickly a tool can decide there is little to do. Before each run: binary preinstalled, cache empty, lockfile absent, node_modules present. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/node_modules/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/node_modules - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/node_modules?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, node_modules [Data set]. https://benchmarks.vlt.io/package-managers/node_modules?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, node_modules}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/node_modules?date=2026-10-20} } ``` # Package managers — cache+lockfile (https://benchmarks.vlt.io/package-managers/cache%2Blockfile) Benchmarks / Package managers / cache+lockfile - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Cache and lockfile both warm. No node_modules to reuse. Before each run: binary preinstalled, cache present, lockfile present, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Blockfile/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Blockfile - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/cache%2Blockfile?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, cache+lockfile [Data set]. https://benchmarks.vlt.io/package-managers/cache%2Blockfile?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, cache+lockfile}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/cache%2Blockfile?date=2026-10-20} } ``` # Package managers — cache+node_modules (https://benchmarks.vlt.io/package-managers/cache%2Bnode_modules) Benchmarks / Package managers / cache+node_modules - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Cache and node_modules present, with no lockfile to pin the graph. Before each run: binary preinstalled, cache present, lockfile absent, node_modules present. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Bnode_modules/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Bnode_modules - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/cache%2Bnode_modules?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, cache+node_modules [Data set]. https://benchmarks.vlt.io/package-managers/cache%2Bnode_modules?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, cache+node_modules}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/cache%2Bnode_modules?date=2026-10-20} } ``` # Package managers — lockfile+node_modules (https://benchmarks.vlt.io/package-managers/lockfile%2Bnode_modules) Benchmarks / Package managers / lockfile+node_modules - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Lockfile and node_modules present, with a cold cache. Before each run: binary preinstalled, cache empty, lockfile present, node_modules present. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/lockfile%2Bnode_modules/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/lockfile%2Bnode_modules - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/lockfile%2Bnode_modules?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, lockfile+node_modules [Data set]. https://benchmarks.vlt.io/package-managers/lockfile%2Bnode_modules?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, lockfile+node_modules}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/lockfile%2Bnode_modules?date=2026-10-20} } ``` # Package managers — cache+lockfile+node_modules (https://benchmarks.vlt.io/package-managers/cache%2Blockfile%2Bnode_modules) Benchmarks / Package managers / cache+lockfile+node_modules - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Everything warm. The best case, and usually dominated by how fast a tool can decide it has nothing to do. Before each run: binary preinstalled, cache present, lockfile present, node_modules present. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Blockfile%2Bnode_modules/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/cache%2Blockfile%2Bnode_modules - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/cache%2Blockfile%2Bnode_modules?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, cache+lockfile+node_modules [Data set]. https://benchmarks.vlt.io/package-managers/cache%2Blockfile%2Bnode_modules?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, cache+lockfile+node_modules}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/cache%2Blockfile%2Bnode_modules?date=2026-10-20} } ``` # Package managers — ci (https://benchmarks.vlt.io/package-managers/ci) Benchmarks / Package managers / ci - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario The tool's CI-oriented install — npm ci and its equivalents. Requires a lockfile and installs strictly from it. Before each run: binary preinstalled, cache empty, lockfile present, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/ci/history ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### DNF and coverage - 2026-10-19: 4 of 6 fixtures published ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Babylon.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Large ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Next.js ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` #### Svelte ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published #### Vue ```csv date 2026-10-14 2026-10-15 2026-10-16 2026-10-17 2026-10-18 2026-10-19 ``` ##### DNF and coverage - 2026-10-19: 0 of 1 fixtures published ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/ci - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/ci?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, ci [Data set]. https://benchmarks.vlt.io/package-managers/ci?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, ci}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/ci?date=2026-10-20} } ``` # Package managers — ci+setup (https://benchmarks.vlt.io/package-managers/ci%2Bsetup) Benchmarks / Package managers / ci+setup - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean install time across 6 fixtures, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (astro, babylon, large, next, svelte, vue) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario This scenario is published but has no description yet. Published in this run, and not yet in any curated ordering. Before each run: binary preinstalled, cache empty, lockfile present, node_modules removed. ## Leaderboard Ranked by mean install time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Install time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean install time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/package-managers/ci%2Bsetup/history ```csv date ``` ### Total install time by tool Mean across all 6 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total install time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total install time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Per-package install time Derived: install time divided by the number of packages the tool installed. A tool with a time but no published package count is left out rather than divided by a guess. ### Per-package time by tool Milliseconds per installed package, mean across fixtures, fastest first; only tools with a value on every fixture. | Tool | Per package | Against the baseline | | --- | --- | --- | ### Per-package time by fixture (ms per installed package, derived) Each fixture's per-package time, run by run, one row per calendar day; an empty cell is no value for that tool on that day. #### Astro ```csv date ``` #### Babylon.js ```csv date ``` #### Large ```csv date ``` #### Next.js ```csv date ``` #### Svelte ```csv date ``` #### Vue ```csv date ``` ### Per-package time data | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | | --- | --- | --- | --- | --- | --- | --- | ## Install footprint ### Package count Packages installed. A range after a count means that tool's installs did not all produce the same number. | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ### Spawned processes | Tool | Astro | Babylon.js | Large | Next.js | Svelte | Vue | Total | | --- | --- | --- | --- | --- | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/package-managers/ci%2Bsetup - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/package-managers/ci%2Bsetup?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Package managers, ci+setup [Data set]. https://benchmarks.vlt.io/package-managers/ci%2Bsetup?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Package managers, ci+setup}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/package-managers/ci%2Bsetup?date=2026-10-20} } ``` # Task runners — average (https://benchmarks.vlt.io/task-runners/average) Benchmarks / Task runners / average - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean execution time across 2 fixtures, averaged over 3 scenarios (derived, not measured), on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (next, run) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Every tool across all of this benchmark's scenarios at once. Mean of, per fixture: Next.js: build, build-cache; Run: run. ## Leaderboard Ranked by mean execution time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Execution time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean execution time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/task-runners/average/history ```csv date ``` ### Total execution time by tool Mean across all 2 fixtures, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total execution time by fixture Each fixture's tools, fastest first. A tool is listed on every fixture it finished; a failed run is named, not timed. ### Total execution time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Next.js | Run | | --- | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/task-runners/average - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/task-runners/average?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Task runners, average [Data set]. https://benchmarks.vlt.io/task-runners/average?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Task runners, average}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/task-runners/average?date=2026-10-20} } ``` # Task runners — build (https://benchmarks.vlt.io/task-runners/build) Benchmarks / Task runners / build - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean execution time across 1 fixture, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (next) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario A project build after install. Measures the build, not the install. ## Leaderboard Ranked by mean execution time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Execution time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean execution time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/task-runners/build/history ```csv date ``` ### Total execution time by tool Mean across the fixture, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total execution time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Next.js | | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/task-runners/build - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/task-runners/build?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Task runners, build [Data set]. https://benchmarks.vlt.io/task-runners/build?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Task runners, build}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/task-runners/build?date=2026-10-20} } ``` # Task runners — build-cache (https://benchmarks.vlt.io/task-runners/build-cache) Benchmarks / Task runners / build-cache - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean execution time across 1 fixture, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (next) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario The same build with a warm build cache. ## Leaderboard Ranked by mean execution time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Execution time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean execution time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/task-runners/build-cache/history ```csv date ``` ### Total execution time by tool Mean across the fixture, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total execution time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Next.js | | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/task-runners/build-cache - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/task-runners/build-cache?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Task runners, build-cache [Data set]. https://benchmarks.vlt.io/task-runners/build-cache?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Task runners, build-cache}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/task-runners/build-cache?date=2026-10-20} } ``` # Task runners — run (https://benchmarks.vlt.io/task-runners/run) Benchmarks / Task runners / run - Run: 2026-10-20 (the latest run). Results regenerate nightly. - Measured: mean execution time across 1 fixture, on GitHub-hosted ubuntu-24.04-arm runners. - Filters in effect: - fixture: all (run) - trend window: last 30 days, 2026-09-22 to 2026-10-20 - trend metric: total time - hidden tools: none - excluded runs: none ## Headline No completed runs were published for this scenario. ## Scenario Script execution only: process startup, reading package.json, and spawning the command. ## Leaderboard Ranked by mean execution time; lower is faster. ±RSD is the pooled relative standard deviation. Since last run compares with the previous run in the trend window. | # | Tool | Key | Mean (s) | ±RSD | Against the baseline | Since last run | Fixtures | Version | | --- | --- | --- | --- | --- | --- | --- | --- | --- | ## Execution time ### Trend by tool Each tool's mean on this run, and how it moved from its first to its last run in the trend window; lower is faster. | Tool | Mean | Change in window | From | To | | --- | --- | --- | --- | --- | ### Performance over time Mean execution time in seconds, one row per calendar day from 2026-09-22 to 2026-10-20. An empty cell is a day with no published value for that tool: no run, a DNF, a partial run or an exclusion. The chart bridges those with a dotted line by default; nothing is interpolated here. For other windows use the history endpoint: https://benchmarks.vlt.io/api/v1/task-runners/run/history ```csv date ``` ### Total execution time by tool Mean across the fixture, fastest first; only tools that completed every fixture. | Tool | Time | Against the baseline | | --- | --- | --- | ### Total execution time data Mean seconds per fixture. DNF: the run exited non-zero and is excluded from every figure. —: not run on that fixture. | Tool | Run | | --- | --- | ## Caveats - Total time is the measurement. Per-package time, change since start and relative-to-baseline are derived from it and labelled derived wherever they appear. ## Method Each tool is configured to behave as much like its peers as possible, but standardisation has limits — tools differ in what they install by default, and in which security checks and validations they run. So the suite counts the packages each tool actually installed, after installation, and publishes both figures: total install time, and time per package. A tool that installs fewer packages finishes sooner without necessarily being faster, which is why neither number is reported alone. ## Environment - Workflow label: `ubuntu-24.04-arm` - VM: `Linux (Ubuntu 24.04)` - Architecture: `arm64` - Processor (CPU): `4 cores` - Memory (RAM): `16 GB` - Storage (SSD): `14 GB` The label is read from the workflow. The remaining figures are GitHub's published specification for standard public runners, not measurements taken here. ## Data & source - This report as JSON: https://benchmarks.vlt.io/api/v1/task-runners/run - API description: https://benchmarks.vlt.io/openapi.json - Index of every page: https://benchmarks.vlt.io/llms.txt - Source code: https://github.com/vltpkg/benchmarks - Methodology: https://github.com/vltpkg/benchmarks/blob/main/docs/METHODOLOGY.md - Methodology slides: https://docs.google.com/presentation/d/1ojXF4jb_1MyGhew2LCbdrZ4e_0vYUr-7CoMJLJsHwZY/edit?usp=sharing - Runner specification: https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories ## Cite - URL: https://benchmarks.vlt.io/task-runners/run?date=2026-10-20 - APA: vlt technology inc. (2026, October 20). vlt benchmarks: Task runners, run [Data set]. https://benchmarks.vlt.io/task-runners/run?date=2026-10-20 ```bibtex @misc{vlt-benchmarks-2026-10-20, title = {vlt benchmarks: Task runners, run}, author = {{vlt technology inc}}, year = {2026}, month = oct, note = {Run of 2026-10-20; results regenerate nightly on GitHub-hosted ubuntu-24.04-arm runners.}, url = {https://benchmarks.vlt.io/task-runners/run?date=2026-10-20} } ```