-
Notifications
You must be signed in to change notification settings - Fork 20
/
Copy pathCHANGES.md
2386 lines (1668 loc) · 92.7 KB
/
CHANGES.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
This is the changelog for Nextstrain CLI. All notable changes in a release
will be documented in this file.
This changelog is intended for _humans_ and follows many of the principles from
[Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
Versions for this project follow the [Semantic Versioning
rules](https://semver.org/spec/v2.0.0.html). Each heading below is a version
released to [PyPI](https://pypi.org/project/nextstrain-cli/) and the date it
was released. The "__NEXT__" heading below describes changes in the unreleased
development source code and as such may not be routinely kept up to date.
# __NEXT__
This release adds explicit (i.e. tested) support for Python version 3.12.
([#369](https://github.com/nextstrain/cli/pull/369))
Note that this Python version support only matters if you're installing
Nextstrain CLI from PyPI or Bioconda
([c.f.](https://docs.nextstrain.org/projects/cli/en/__NEXT__/installation/)). It
does not apply if you're installing Nextstrain CLI using the standalone
installation method we recommend in the [Nextstrain installation
documentation](https://docs.nextstrain.org/page/install.html). In that case, a
supported Python version is always bundled with `nextstrain`.
## Improvements
* The Conda runtime now uses Micromamba 1.5.8 (upgraded from 1.1.0) to manage
the runtime environment. The newer version supports Zstandard-compressed
index files which speeds up `nextstrain setup` and `nextstrain update` for
the Conda runtime.
([#367](https://github.com/nextstrain/cli/pull/367))
# 8.3.0 (30 April 2024)
## Improvements
* Commands which utilize a runtime—`nextstrain build`, `nextstrain shell`, and
`nextstrain view`—now support specifying envdirs to forward into the runtime
environment by setting `NEXTSTRAIN_RUNTIME_ENVDIRS` to a `:`-separated (`;`
on Windows) list of paths. This is in addition to the existing support for
specifying one or more `--envdir` options.
([#365](https://github.com/nextstrain/cli/pull/365))
# 8.2.0 (6 February 2024)
## Features
* `nextstrain build` and `nextstrain shell` now better support pathogen
repositories which place workflows in subdirectories. The top-level of the
repo must contain a `nextstrain-pathogen.yaml` file for this support to
activate. The file may be empty for now, though we anticipate using it for
pathogen-level metadata in the future to aid indexing, listing, and
attribution of pathogen repos.
As an example of the new support, consider the following repo layout
mpox/
├── nextstrain-pathogen.yaml
├── ingest/
│ ├── Snakefile
│ └── …
├── phylogenetic/
│ ├── Snakefile
│ └── …
├── shared/
│ ├── reference.fasta
│ └── …
└── …
where `ingest/` and `phylogenetic/` contain workflows that use
`shared/reference.fasta` via a relative path (i.e.
`../shared/reference.fasta`).
It's now possible to invoke those workflows with any of the following:
nextstrain build mpox/ingest/
nextstrain build mpox/phylogenetic/
cd mpox
nextstrain build ingest/
nextstrain build phylogenetic/
cd phylogenetic
nextstrain build .
nextstrain build ../ingest/
regardless of runtime.
Previously, such workflows required careful invocation, e.g.
nextstrain build mpox/ -d phylogenetic/ -s phylogenetic/Snakefile
when using runtimes with filesystem isolation (i.e. the [containerized][]
ones; Docker, Singularity, and AWS Batch) but not when using runtimes without
it.
When active, this feature makes the top-level of the pathogen repo (e.g.
`mpox/`) available in the container at `/nextstrain/build` while the
initial working directory is set to the workflow subdirectory in the
container (e.g. `/nextstrain/build/phylogenetic`). That is, the filesystem
isolation boundary is drawn at the top-level of the pathogen repo instead of
at the workflow directory (i.e. what's given to `nextstrain build`).
([#355](https://github.com/nextstrain/cli/pull/355))
[containerized]: https://docs.nextstrain.org/projects/cli/en/8.2.0/runtimes/#comparison
## Improvements
* We now produce standalone installation archives for macOS running on aarch64
hardware (aka arm64, Apple Silicon, M1/M2). The standalone installer will
use these archives starting with this release.
([#357](https://github.com/nextstrain/cli/pull/357),
[#358](https://github.com/nextstrain/cli/pull/358))
* The Conda and Docker runtime checks performed by `nextstrain setup` and
`nextstrain check-setup` now test if Rosetta 2 is enabled for macOS on
aarch64 (aka arm64, Apple Silicon, M1/M2) hardware. Rosetta is required for
the Conda runtime and optional, but recommended, for the Docker runtime.
Previously only the standalone installer checked for Rosetta, but starting
with this release it will not.
([#361](https://github.com/nextstrain/cli/pull/361),
[#358](https://github.com/nextstrain/cli/pull/358))
* `nextstrain build` now errors if a [development overlay option][] such as
`--augur` or `--auspice` is given when using a runtime without support for
those (anything but Docker or Singularity). Previously, it would silently
ignore those options when unsupported by the runtime. The new behaviour
matches the behaviour of `nextstrain shell` since 5.0.0.
([#354](https://github.com/nextstrain/cli/pull/354))
[development overlay option]: https://docs.nextstrain.org/projects/cli/en/8.2.0/commands/build/#development-options-for-docker
# 8.0.1 (29 January 2024)
## Improvements
* Several kinds of errors from `nextstrain login` and `nextstrain whoami`
related to their interactions with a remote server are now clearer.
([#347](https://github.com/nextstrain/cli/pull/347))
# 8.0.0 (18 January 2024)
This release drops support for Python versions 3.6 and 3.7 and adds support for
3.11. ([#325](https://github.com/nextstrain/cli/pull/325),
[#327](https://github.com/nextstrain/cli/pull/327))
Note that this Python version support only matters if you're installing
Nextstrain CLI from PyPI or Bioconda
([c.f.](https://docs.nextstrain.org/projects/cli/en/8.0.0/installation/)). It
does not apply if you're installing Nextstrain CLI using the standalone
installation method we recommend in the [Nextstrain installation
documentation](https://docs.nextstrain.org/page/install.html). In that case, a
supported Python version is always bundled with `nextstrain`.
## Improvements
* The `nextstrain remote` family of commands now support alternative
nextstrain.org-like remotes such as internal Nextstrain Groups Server
instances and development instances of nextstrain.org. Authentication with
these remotes is supported via `nextstrain login` and related commands.
Remotes maintain their authentication alongside each other, e.g. you can be
logged into nextstrain.org as well as an alternative nextstrain.org-like
instance.
As an example, a Nextstrain Groups Server instance accessible at
nextstrain.example.com could now be logged into and interacted with like so:
nextstrain login nextstrain.example.com
nextstrain whoami nextstrain.example.com
nextstrain remote ls nextstrain.example.com/groups/bedford-lab/
The default remote is still nextstrain.org.
([#333](https://github.com/nextstrain/cli/pull/333))
* `nextstrain login` now performs authentication via a web browser by default
(using OpenID Connect 1.0 and OAuth 2.0). The previously method of direct
password entry is still used when a username is provided (e.g. with
`--username` or `-u`). See `nextstrain login --help` for more information.
([#333](https://github.com/nextstrain/cli/pull/333))
* With the new support for being logged into multiple remotes, `nextstrain
logout` now also supports an `--all` flag to remove all locally-saved
credentials without having to logout of each remote separately.
([#333](https://github.com/nextstrain/cli/pull/333))
* `nextstrain remote upload` now skips gzip compression when uploading
zstandard-compressed (`.zst`) files, matching its behaviour for other types
of compressed files.
([#330](https://github.com/nextstrain/cli/pull/330))
* Commands that may automatically open a web browser, e.g. `nextstrain view` or
`nextstrain login`, now respect the semi-conventional `NOBROWSER` environment
variable to disable opening a browser. Set `NOBROWSER` to a non-empty value
to disable automatic opening. When disabled, the URL to manually open will
be shown.
([#332](https://github.com/nextstrain/cli/pull/332))
* The error message emitted by the `nextstrain remote` family of commands when
they're unable to access an S3 bucket now lists a few possible reasons for
the failure.
([#341](https://github.com/nextstrain/cli/pull/341))
* `nextstrain remote download` now supports downloading core datasets which are
only visible on the web by direct access via their URL. For example, the
following now work (where in previous versions they did not):
nextstrain remote download https://nextstrain.org/nextclade/sars-cov-2/21L
nextstrain remote download https://nextstrain.org/enterovirus/d68/vp1/2020-01-23
This support also covers [past snapshots of
datasets](https://docs.nextstrain.org/en/latest/guides/versions.html), which
is a recently-added feature to nextstrain.org. For example:
nextstrain remote download https://nextstrain.org/flu/seasonal/h3n2/ha/6y@2023-07-01
nextstrain remote download https://nextstrain.org/ncov/gisaid/21L/global/6m@2024-01-09
([#345](https://github.com/nextstrain/cli/pull/345))
## Bug fixes
* Commands making use of user authentication (e.g. `nextstrain login`,
`nextstrain whoami`, some invocations of `nextstrain remote`, etc.) no longer
fail if the logged in user is a member of no groups.
([#323](https://github.com/nextstrain/cli/pull/323))
* On Linux and macOS, the standalone installer's final instructions now
correctly suggest modifying `~/.profile` instead of, incorrectly, `~/.shrc`
when the default shell is sh.
([#321](https://github.com/nextstrain/cli/pull/321))
* The Singularity runtime once again supports Apptainer's `singularity`
executable. The minimum version checking added in 7.0.0 broke usage of the
Singularity runtime with Apptainer (compared with SingularityCE). Our intent
is to support both lineages of Singularity. Thanks to @osageorange for
raising this issue and testing the fix!
([#343](https://github.com/nextstrain/cli/pull/343))
# 7.4.0 (21 September 2023)
## Improvements
* When attached to an AWS Batch job, `nextstrain build` now only requires
confirmation to stop the job (e.g. a double Control-C/`SIGINT`) when stdin is
a terminal (TTY). When stdin is not a terminal, as is frequently the case in
automated or programmatic contexts, then `SIGINT` requests job cancellation
immediately.
([#308][])
* When attached to an AWS Batch job, `nextstrain build` now also detaches upon
receiving `SIGHUP` (in addition to detaching upon receiving
Control-Z/`SIGTSTP`). `SIGHUP` is sent, for example, when a user closes (or
loses) the terminal in which `build` is still running.
([#308][])
* When launching or attaching to an AWS Batch job, `nextstrain build` now
supports the `--detach-on-interrupt` option to change Control-C/`SIGINT` to a
request to detach from the job rather than to cancel it. This is useful in
automated contexts where `SIGINT` may be sent outside of a user's control,
and it's also handy as a molly-guard to avoid accidental cancellation when
the intent is to just observe a job.
([#308][])
* When attaching to an AWS Batch job, `nextstrain build` now supports the
`--cancel` option to immediately request cancellation.
([#308][])
[#308]: https://github.com/nextstrain/cli/pull/308
## Bug fixes
* We've plugged some isolation leaks in the Conda runtime where the
[Python user site directory](https://docs.python.org/3/library/site.html),
e.g. `~/.local/lib/pythonX.Y/site-packages`, as well as the
[`PYTHONPATH` and `PYTHONHOME` environment variables](https://docs.python.org/3/using/cmdline.html#environment-variables)
could influence and break the runtime.
([#311](https://github.com/nextstrain/cli/pull/311))
# 7.3.0.post1 (19 September 2023)
_See also changes in 7.3.0 which was an unreleased version._
## Development
* Update CI to test against the SingularityCE 3.x series only ([#314](https://github.com/nextstrain/cli/pull/314))
# 7.3.0 (19 September 2023)
_Unreleased due to [test failures](https://github.com/nextstrain/cli/actions/runs/6238951959). Changes first released as part of 7.3.0.post1._
## Improvements
* build: Providing a path to a pathogen build directory is no longer required
when the AWS Batch runtime is in use (e.g. with `--aws-batch`) and both the
`--attach` and `--no-download` options are given. This allows usages which
just want to check job status or logs to stop providing a meaningless/unused
directory.
([#305](https://github.com/nextstrain/cli/pull/305))
* The initial setup of the Conda runtime, i.e. with `nextstrain setup conda`,
now explicitly installs the latest available runtime package version. This
was the implicit intent and expectation, but, for various Conda packaging
reasons, it was (silently) not always the result. Now an error will occur if
the latest version is not installable. This matches the existing behaviour
when updating the Conda runtime, i.e. with `nextstrain update conda`.
([#312](https://github.com/nextstrain/cli/pull/312))
## Bug fixes
* The `HOST` and `PORT` environment variables are now respected by `nextstrain
view`. They provide defaults when `--host` and/or `--port` aren't provided.
([#310](https://github.com/nextstrain/cli/pull/310))
* Updated fsspec and s3fs dependencies to avoid version 2023.9.1, since they
caused `nextstrain build --aws-batch` invocations to fail.
([#313](https://github.com/nextstrain/cli/pull/313))
# 7.2.0 (17 August 2023)
## Bug fixes
* The Conda runtime now uses Micromamba 1.1.0 (upgraded from 1.0.0) to manage
the runtime environment. The newer version has improved error handling in
edge cases where the previous version silently failed when used during
`nextstrain update`.
([#301](https://github.com/nextstrain/cli/pull/301))
* The Singularity runtime now cleans up unnecessary caches during `nextstrain
update` and `nextstrain setup`. This reduces the storage space required by
the runtime and prevents it from growing unbounded over time. However, this
also results in increases in network transfer during `nextstrain update`. We
may mitigate the additional transfer in the future.
([#303][])
## Documentation
* The command-line `--help` output for commands and the corresponding
documentation pages on the web are more integrated and improved in various
small ways. In particular, command options are more cross-referencable and
directly linkable and the wrap-width of `--help` output is more consistent
and reliably readable.
([#299](https://github.com/nextstrain/cli/pull/299))
* We've added a large section of new and expanded [documentation on
runtimes](https://docs.nextstrain.org/projects/cli/page/runtimes/), both in
general and for each individual runtime.
([#303][])
* We've added and expanded documentation about configuration, including the
[config file](https://docs.nextstrain.org/projects/cli/page/config/file/) and
the [paths](https://docs.nextstrain.org/projects/cli/page/config/paths/) used
by Nextstrain CLI. Documentation in general now includes more about the
environment variables that can be used to configure Nextstrain CLI.
([#303][])
[#303]: https://github.com/nextstrain/cli/pull/303
# 7.1.0 (22 June 2023)
## Improvements
* Commands that use a runtime (`nextstrain build`, `nextstrain shell`, and
`nextstrain view`) now support two new options for setting or passing thru
environment variables into the runtime environment:
--env <name>[=<value>]
--envdir <path>
When either of these options are given, the default behaviour of
automatically passing thru several "well-known" environment variables is
disabled. That is, the following "well-known" environment variables are only
automatically passed thru when the new `--env` and `--envdir` options are
_not_ used:
- `AUGUR_RECURSION_LIMIT`
- `AUGUR_MINIFY_JSON`
- `AWS_ACCESS_KEY_ID`
- `AWS_SECRET_ACCESS_KEY`
- `AWS_SESSION_TOKEN`
- `ID3C_URL`
- `ID3C_USERNAME`
- `ID3C_PASSWORD`
- `RETHINK_HOST`
- `RETHINK_AUTH_KEY`
Pass these variables explicitly via `--env` or `--envdir` if you need them in
combination with other `--env` or `--envdir` usage. For more usage details,
use the `--help-all` flag of any of those commands, e.g. `nextstrain build
--help-all`.
([#289](https://github.com/nextstrain/cli/pull/289))
* Environment variables are now passed to the Docker and AWS Batch runtimes via
more secure means when the container image in use is new enough to support it
(`nextstrain/base:build-20230613T204512Z` and newer). This ensures the env
values aren't visible in the container's config (e.g. via `docker inspect`,
`aws batch describe-jobs`, the AWS web console). If you're using an older
image, you can update it with `nextstrain update docker`.
For Docker, environment variables are written to an internal and temporary
envdir directory visible only to the current user which is deleted
immediately after use at container start.
For AWS Batch, environment variables are written to a ZIP archive on S3,
alongside but separate from the ZIP archive of the build dir. This env
archive is deleted from S3 immediately after use at container start.
Both of these approaches minimize the amount of time environment variable
values exist outside of memory, persisted to storage (disk, S3).
([#289](https://github.com/nextstrain/cli/pull/289))
## Bug fixes
* `nextstrain view` now waits (up to 10s) for Auspice to start responding
before automatically opening it in the browser. This should eliminate the
previous behaviour of sometimes opening the browser too soon.
([#291](https://github.com/nextstrain/cli/pull/291))
# 7.0.1 (31 May 2023)
## Bug fixes
* `nextstrain update` for the Conda runtime no longer reports an "invalid
version" error. This was a regression introduced in 7.0.0.
([#286](https://github.com/nextstrain/cli/pull/286))
# 7.0.0 (26 May 2023)
This release is mostly a bug fix release for our Conda and Singularity
runtimes. However, it contains a **potentially-breaking change** for existing
usages of the Singularity runtime: **the minimum required Singularity version
has changed from 2.6.0 to 3.0.0**. This change was required for a critical bug
fix. If you do not use the Singularity runtime, there are no
potentially-breaking changes in this release.
## Improvements
* `nextstrain shell` now notes which runtime is being entered in its initial
messaging to establish more context for the user (and for developers when
troubleshooting).
([#283][])
* The Singularity runtime now checks for the minimum required Singularity
version (3.0.0 with this release) during `nextstrain check-setup`.
([#283][])
## Bug fixes
* Setup and upgrade of the Conda runtime now only uses stable "main" channel
releases when determining the latest release version, as intended.
Previously, testing and development releases could be selected if they were
newer than the last stable release. Additionally, if there are multiple
builds for a release version, the highest numbered build (i.e. newest) is now
used instead of the lowest.
([#280](https://github.com/nextstrain/cli/pull/280))
* The Singularity runtime now works with our container runtime images from
`build-20230411T103027Z` onwards. The Snakemake upgrade in that image
version resulted in "read-only file system" errors which referenced the
user's home directory. Those errors are now fixed.
([#283][])
* The prompt for `nextstrain shell`—a stylized variant of the Nextstrain
wordmark—now works when using the Singularity runtime regardless of
Singularity version. Previously Singularity's default prompt of
`Singularity> ` overrode ours when using Singularity versions ≥3.5.3.
([#283][])
* More robust command-line processing is used for the Singularity runtime on
Singularity versions ≥3.10.0. Singularity's early (and unexpected)
evaluation of arguments that look like (but aren't) shell variable
substitutions is disabled.
([#283][])
## Development
* The command lines and environment overrides of many (but not all) process
invocations are now logged when `NEXTSTRAIN_DEBUG` is enabled.
([#283][])
[#283]: https://github.com/nextstrain/cli/pull/283
# 6.2.1 (24 March 2023)
## Bug fixes
* We've fixed and future-proofed a compatibility bug with a third-party library
that can occur under very specific conditions when `nextstrain build` submits
AWS Batch jobs.
([#261](https://github.com/nextstrain/cli/pull/261))
* The update process for the Conda runtime is now more robust and less likely
to get stuck at an old version.
([#266](https://github.com/nextstrain/cli/pull/266))
# 6.2.0 (28 February 2023)
## Improvements
* `nextstrain build --aws-batch --attach …` no longer offers to cancel (via
Control-C) or detach (via Control-Z) from the job if it's already complete.
Instead, Control-C will exit the program without delay and without trying to
cancel the job.
([#253][])
* `nextstrain build` now supports a `--no-logs` option to suppress the fetching
and printing of job logs when attaching to a completed AWS Batch build. As
log fetching can often take longer than a selective download of the results
(i.e. via `--download`), this is a time (and terminal scrollback) saver when
all you want are a few of the results files.
([#253][])
[#253]: https://github.com/nextstrain/cli/pull/253
## Bug fixes
* An error message that's printed by `nextstrain remote upload` when unknown
files are given for upload to destinations on nextstrain.org now properly
includes the actual list of unknown files instead of the placeholder
`{files}`.
([#260](https://github.com/nextstrain/cli/pull/260))
* When running on Python ≥3.10, the `--help` output of `nextstrain build`,
`nextstrain view`, and `nextstrain shell` once again shows just the most
common options. All options are still shown with `--help-all`. A regression
since Python 3.10 meant that `--help` acted the same as `--help-all` before
this fix. This affected any installation on Python ≥3.10, including
standalone installations, since the standalone binaries bundle Python 3.10.
([#259](https://github.com/nextstrain/cli/pull/259))
# 6.1.0.post1 (18 January 2023)
## Documentation
* Minor improvements to the way we list and describe our computing platforms.
# 6.1.0 (18 January 2023)
## Improvements
* We've added a new Singularity runtime based on our existing Docker runtime.
Singularity is a container system freely-available for Linux platforms. It
is commonly available on institutional HPC systems as an alternative to
Docker, which is often not supported on such systems. When you use
Singularity with the Nextstrain CLI, you don't need to install any other
Nextstrain software dependencies as validated versions are already bundled
into a container image by the Nextstrain team.
Run `nextstrain setup singularity` to get started.
([#248](https://github.com/nextstrain/cli/pull/248))
# 6.0.3 (17 January 2023)
## Improvements
* The output of `nextstrain check-setup` for the Conda runtime will now hint
about running `nextstrain setup conda` first if the runtime seems supported
but not yet set up.
([#250][])
## Documentation
* Documentation and `--help` output now standardizes on the term _runtime_ to
describe the various ways Nextstrain CLI provides access to the Nextstrain
software tools. Previously we used a mix of _build environment_, _runner_,
and _runtime_ in user-facing messages. This brings Nextstrain CLI in line
with the rest of our documentation.
([#250][])
* The installation documentation now includes the standalone installer as well
as installing from Bioconda. It now covers all the ways our releases are
distributed.
([#250][])
* The output of `nextstrain --help` now notes how to find command-specific
usage info and our online documentation.
([#250][])
* An out-of-date description in `nextstrain build --help` has been revised and
updated.
([#250][])
## Development
* A new [glossary page in our documentation](https://docs.nextstrain.org/projects/cli/page/glossary/)
will help keep our shared understanding of terms straight.
([#250][])
* Links to our online documention in the `--help` output of commands will now
omit any [local part](https://peps.python.org/pep-0440/#local-version-identifiers),
e.g. `+git`, of the running CLI version. This makes links correct even when
running development builds.
([#250][])
[#250]: https://github.com/nextstrain/cli/pull/250
# 6.0.2 (3 January 2023)
_See also changes in 6.0.1 which was an unreleased version._
## Bug fixes
* A new deprecation warning from the `cryptography` module (version 39) when
running on Python 3.6 is now suppressed as it's just noise to an end user.
This probably moves us closer to dropping 3.6 support ourselves, but it's not
so onerous yet.
([#246](https://github.com/nextstrain/cli/issues/246))
# 6.0.1 (3 January 2023)
_Unreleased due to [test failures](https://github.com/nextstrain/cli/issues/245). Changes first released as part of 6.0.2._
## Improvements
* The standalone installation archives used by the standalone installer will
now work on even older Linux distributions:
|distro |now |was |
|-------------|-------|-------|
|Ubuntu |14\.04 |18\.04 |
|Debian |8 |10 |
|RHEL/CentOS |7 |8 |
|Fedora |19 |28 |
|OpenSUSE |12\.3 |15\.3 |
If you've previously encountered errors like the following:
/lib64/libc.so.6: version `GLIBC_2.27' not found (required by […]/.nextstrain/cli-standalone/nextstrain)
when using the standalone installer (or standalone archives directly), i.e.:
curl -fsSL --proto '=https' https://nextstrain.org/cli/installer/linux | bash
then this change should resolve that error! The new minimum required glibc
version is 2.17 (was 2.27 previously).
([#243](https://github.com/nextstrain/cli/pull/243))
## Bug fixes
* The automatic opening of a browser tab (or window) by `nextstrain view`—a
feature introduced in the last release (6.0.0)—now also works for standalone
installations.
([#244](https://github.com/nextstrain/cli/pull/244))
# 6.0.0 (13 December 2022)
This release contains a **potentially-breaking change** for existing usages of
`nextstrain view`, though we expect the change to impact very few usages. The
change is described below.
## Improvements
* `nextstrain view` now supports viewing narratives, as was always intended.
Previously the launched Auspice would either show baked in test narratives or
no narratives at all, depending on the Auspice version in the runtime.
([#240][])
* `nextstrain view` now supports being given more kinds of paths, including
paths to a specific dataset or narrative file and paths to directories
containing _auspice/_ and/or _narratives/_ subdirectories.
This is a **potentially-breaking change**, as `nextstrain view <dir>` will
now prefer to show datasets from _`<dir>`/auspice/_ if that subdirectory
exists. Previously it would only ever look for datasets in the given
_`<dir>`_. We expect this to change behaviour for very few usages as it only
affects situations where _`<dir>`_ contains both datasets and an _auspice/_
directory.
See `nextstrain view --help` for more details on the kinds of paths
supported.
([#240][])
* `nextstrain view` now automatically opens Auspice in a new browser tab (or
window) by default when possible.
If a specific dataset or narrative file was given as the path to `nextstrain
view`, then that dataset or narrative is opened. Otherwise, if there's only
a single dataset or narrative available in the directory path given to
`nextstrain view`, then it is opened. Otherwise, Auspice's listing of
available datasets and narratives is opened.
([#240][])
[#240]: https://github.com/nextstrain/cli/pull/240
* Local images used in a narrative are now automatically embedded into it when
uploading the narrative to nextstrain.org via `nextstrain remote upload`.
In local text editors which can render Markdown, this permits previewing of
narratives that reference images on the local filesystem without requiring
manual conversion to remote images or embedded images before upload.
([#235](https://github.com/nextstrain/cli/pull/235))
* The `nextstrain remote upload` command now outputs a nicer error message
that's more interpretable and actionable when nextstrain.org returns a "bad
request" error. The error message also includes the error details returned
by nextstrain.org.
([#238](https://github.com/nextstrain/cli/pull/238))
## Development
* The Conda runtime now uses Micromamba 1.0.0 (an upgrade from 0.27.0).
([#233](https://github.com/nextstrain/cli/pull/233))
# 5.0.1 (1 November 2022)
## Bug fixes
* `nextstrain shell` no longer errors when its history file, e.g.
_~/.nextstrain/shell-history_, doesn't exist. This primarily affected the
Docker runtime and was a regression from 4.2.0 introduced in 5.0.0.
([#232](https://github.com/nextstrain/cli/pull/232))
# 5.0.0 (25 October 2022)
_Version 5.0.0 had two development pre-releases (5.0.0.dev0 and 5.0.0.dev1)
prior to final release. For convenience, the changes from those pre-releases
are also re-described here._
The major improvement in this release is the introduction of a new Conda
runtime, filling a gap between the Docker runtime and the ambient runtime
(formerly "native" runtime). See more details below.
This release also contains **potentially-breaking changes** for existing
usages of `nextstrain remote download` and `nextstrain update`. The changes
are described below.
## Improvements
* A new Conda runtime (aka runner or build environment) now complements the
existing Docker and ambient runtimes and fills a gap between them. This
runtime is more isolated and reproducible than your ambient environment, but
is less isolated and robust than the Docker runtime. Like the Docker
runtime, the Conda runtime is fully-managed by Nextstrain CLI and receives
updates via `nextstrain update`.
The new runtime uses the [Conda](https://docs.conda.io) ecosystem with
packages from [our own channel](https://anaconda.org/Nextstrain/nextstrain-base),
[Bioconda](https://bioconda.github.io/) and
[Conda-Forge](https://conda-forge.org/), installed by
[Micromamba](https://mamba.readthedocs.io/en/latest/user_guide/micromamba.html)
into an isolated location, typically `~/.nextstrain/runtimes/conda/env`. It
does not interact with or impact other usage of Conda/Mamba environments and
will not, for example, appear in the output of `conda env list`.
Set up of the runtime is automated and can be performed by running:
nextstrain setup conda
When complete, you'll be able to use the `--conda` runtime option supported
by Nextstrain CLI commands such as `nextstrain build`, `nextstrain view`,
`nextstrain shell`, etc.
([#218][])
* The "native" runtime (aka runner or build environment) is now the "ambient"
runtime. This name better reflects what it is and further distinguishes it
from the new Conda runtime, which is also "native" in the binary executable
sense.
Existing usages of "native" should be unaffected. The `--native` option
continues to work anywhere it used to previously, though it is hidden from
`--help` output to discourage new use. The string "native" is also accepted
anywhere runner names are accepted, e.g. in config as the `core.runner`
setting or in command-line arguments to `check-setup` or `setup`.
([#224](https://github.com/nextstrain/cli/pull/224))
* `nextstrain setup docker` now downloads the Docker runtime image if it's not
already available locally. This can be a useful initial step after
installation to avoid the automatic download on first use.
([#222](https://github.com/nextstrain/cli/pull/222))
* The local filenames produced by `nextstrain remote download` now include
more of the remote dataset/narrative path. This reduces the potential for
ambiguous filenames and makes it easier to copy datasets/narratives between
destinations (e.g. from one group to another) while retaining the same path.
It is, however, a **potentially-breaking change** if you're relying on the
filenames of the downloaded datasets/narratives (e.g. for automation).
For example, downloading `nextstrain.org/flu/seasonal/h3n2/ha/2y` previously
produced the local files:
```
2y.json
2y_root-sequence.json
2y_tip-frequencies.json
```
which could easily conflict with the similarly-named
`nextstrain.org/flu/seasonal/h3n2/na/2y`,
`nextstrain.org/flu/seasonal/h1n1pdm/ha/2y`, etc. The downloaded files are
now named:
```
flu_seasonal_h3n2_ha_2y.json
flu_seasonal_h3n2_ha_2y_root-sequence.json
flu_seasonal_h3n2_ha_2y_tip-frequencies.json
```
Within groups, filenames are similarly longer but the group name is not
included. For example, downloading `groups/blab/ncov/cross-species/cat`
previously produced:
```
cat.json
cat_root-sequence.json
cat_tip-frequencies.json
```
and now produces:
```
ncov_cross-species_cat.json
ncov_cross-species_cat_root-sequence.json
ncov_cross-species_cat_tip-frequencies.json
```
([#213](https://github.com/nextstrain/cli/pull/213))
* Advanced globbing features are now supported in patterns for the `--download`
option of `nextstrain build`, including multi-part wildcards (`**`), extended
globbing (`@(…)`, `+(…)`, etc.), and negation (`!…`). Basic globbing
features like single-part wildcards (`*`), character classes (`[…]`), and
brace expansion (`{…, …}`) are still supported. Note that the `--download`
option continues to be applicable only to the AWS Batch runtime (e.g. the
`--aws-batch` option).
([#215](https://github.com/nextstrain/cli/pull/215))
* `check-setup` now accepts one or more runtime names as arguments.
The default behaviour doesn't change, but specifying runtimes now lets you
restrict checks to a single runtime or, with multiple runtimes, re-order them
by your preference for use with --set-default.
([#218][])
* `update` now only updates a specific runtime instead of all of them at once.
With no arguments, the default runtime is updated. The name of another
runtime to update instead may be provided as an argument.
In practice this isn't much of a behaviour change because only one runtime
currently supports updating (Docker); the others (ambient, AWS Batch) just
pass. Existing users are unlikely to notice the change unless they use
multiple runtimes and Docker is not their default. In that case, `update`
may stop updating Docker for them when it would have done so previously,
which is a **potentially-breaking change**.
([#218][])
* A new command, `setup`, now exists to perform automatic set up of runtimes
that support it (currently only Conda). For all runtimes, even those that
don't support automatic set up, the `setup` command will also run the same
checks as `check-setup` and optionally set the default runtime.
([#218][])
* The shell launched by the `shell` command now remembers its own command
history and differentiates its command prompt from other shells with a
stylized variant of the Nextstrain wordmark.
([#218][])
* The output of commands in dry run mode (e.g. with the `--dry-run` option) is
now uniformly indicated to be a dry run by the prefix `DRY RUN │ `. This
includes the `remote` family of commands and the new `setup` command.
([#218][])
* Runtime checks in `check-setup` and `setup` now test for not just the
presence of Snakemake, Augur, and Auspice, but also that they can be
executed.
([#218][])
## Development
* We now provide standalone installers (i.e. shell programs) to download and
unpack the standalone installation archives into standard locations,
potentially upgrading/overwriting a prior standalone install. These
installers will be served from GitHub directly out of this project's
repository via convenience redirects on nextstrain.org.
These will eventually form the basis for Nextstrain install instructions that
don't suffer from Python bootstrapping issues. As a preview for now, you can
play around with the following platform-specific commands:
curl -fsSL --proto '=https' https://nextstrain.org/cli/installer/linux | bash
curl -fsSL --proto '=https' https://nextstrain.org/cli/installer/mac | bash
Invoke-RestMethod https://nextstrain.org/cli/installer/windows | Invoke-Expression
A new companion command, `init-shell`, exists to simplify shell configuration
(i.e. `PATH` modification) for such installations.
* The `NEXTSTRAIN_HOME` environment variable can now be used to specify the
desired location for per-user settings, files, etc., overriding the default
of _~/.nextstrain/_.
([#218][])
* A new `nextstrain authorization` command makes it easier to generate direct
requests to nextstrain.org's web API using the same credentials as the CLI.
([#229](https://github.com/nextstrain/cli/pull/229))
* The development documentation now documents how to build the documentation
locally, and sphinx-autobuild is used to make a very nice edit-preview cycle
with quick turnaround.
([#218][])
* Development dependency issues with `flake8` and `sphinx-markdown-tables`,
caused by upstream changes, are now resolved.
([#218][])
# 5.0.0.dev1 (25 October 2022)
_This is the second development pre-release made prior to the final release of 5.0.0._
## Improvements
* The "native" runtime (aka runner or build environment) is now the "ambient"
runtime. This name better reflects what it is and further distinguishes it
from the new Conda runtime, which is also "native" in the binary executable
sense.
Existing usages of "native" should be unaffected. The `--native` option
continues to work anywhere it used to previously, though it is hidden from
`--help` output to discourage new use. The string "native" is also accepted
anywhere runner names are accepted, e.g. in config as the `core.runner`
setting or in command-line arguments to `check-setup` or `setup`.
([#224](https://github.com/nextstrain/cli/pull/224))
* `nextstrain setup docker` now downloads the Docker runtime image if it's not
already available locally. This can be a useful initial step after
installation to avoid the automatic download on first use.
([#222](https://github.com/nextstrain/cli/pull/222))
* `nextstrain build`'s check for use of the `--image` option with unsupported
runtimes now includes the Conda runtime.
([#224](https://github.com/nextstrain/cli/pull/224))
* The Conda runtime now uses the new [`nextstrain-base` Conda
meta-package](https://anaconda.org/Nextstrain/nextstrain-base) instead of
using a hardcoded list of packages.
This decouples Conda runtime updates from Nextstrain CLI updates, as we can
make new releases of `nextstrain-base` and users can update to those without
upgrading Nextstrain CLI itself. This brings the update story for the Conda
runtime into much better parity with the Docker runtime.
Using the meta-package also brings increased reproducibility to the runtime,
as the package completely locks its full transitive dependency tree. This
means that if version _X_ of `nextstrain-base` worked in the past, it'll
still work the same way in the future.
The `NEXTSTRAIN_CONDA_BASE_PACKAGE` environment variable may be used with
`nextstrain setup conda` to install a specific version. The value is a
[Conda package specification][], e.g. `nextstrain-base ==X`.
([#228](https://github.com/nextstrain/cli/pull/228))
* The Conda runtime now uses a pinned version of Micromamba (currently 0.27.0)
so that new releases of the latter can't break `nextstrain setup conda` or
`nextstrain update conda` between one day and the next. The pinned version
will be bumped up over time as needed with subsequent releases of Nextstrain
CLI.
The `NEXTSTRAIN_CONDA_MICROMAMBA_VERSION` environment variable may be used
with `nextstrain setup conda` to override the built-in pin, either with
another specific version or `latest`.
[Conda package specification]: https://docs.conda.io/projects/conda/en/latest/user-guide/concepts/pkg-specs.html#package-match-specifications
## Bug fixes
* The Conda runtime now runs Micromamba in greater isolation to avoid undesired
interactions when a) Nextstrain CLI itself is running inside an
externally-activated Conda environment and/or b) user-specific Mamba
configuration exists. This applies to usages of `nextstrain setup` and
`nextstrain update` with the Conda runtime.
([#223](https://github.com/nextstrain/cli/pull/223))
* The Conda runtime now configures the appropriate channels during `update` too,
not just during `setup`, ensuring package updates are found.
([#228](https://github.com/nextstrain/cli/pull/228))
* The Conda runtime now avoids pinning Python in the isolated environment to
allow it to be upgraded by `update`.