Recent releases for jq

jq jq-1.5rc1

2015-01-01T02:54:01+00:00

Salient new features since 1.4: - regexp support (using Oniguruma) - an online streaming parser Included is the ability to control reading of inputs from the jq program, using the new `input` and `inputs` builtins. Finally we can write: jq -n 'reduce inputs as $i ( ... )' # reduce online! to reduce inputs without slurping them first. This works with streaming too. - try/catch, for catching and handling errors (this makes for a dynamic non-local exit system) - a lexical non-local exit system Using try/catch to break out of control structures was not a good thing. A lexical mechanism is. You can now say label $foo | ..... | break $foo where the break causes control to return to the label $foo, which then produces `empty` (backtracks). There's named and anonymous labels. - tail call optimization (TCO), which allows efficient recursion in jq - a variety of new control structure builtins (e.g., `while(cond; exp)`, `repeat(exp)`, `until(cond; next)`) - an enhanced form of `reduce`: `foreach exp as $name (init_exp; update_exp; extract_exp)` - a proper module system import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::\* namespace - the ability to read module data files import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar - `--argjson var ''` Using --arg var bit me too many times :) - `--in-place` / `-i` for in-place editing of files - support for application/json-seq. - a variety of new utility functions, many being community contributions - a variety of performance enhancements (e.g., constant folding) - `def($a): ...;` is now allowed as an equivalent of `def(a): a as $a | ...;` Lastly, don't forget the wiki! It has a lot of new content since 1.4, much of it contributed by the community.

jq jq-1.5rc2

2015-07-27T04:14:01+00:00

Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki. We're nearing a 1.5 release, finally. Salient new features since 1.4: - regexp support (using Oniguruma)! - a proper module system `import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace` and `include "foo/bar"; # import foo/bar.jq's defs into the top-level` - destructuring syntax (`. as [$first, $second, {$foo, $bar}] | ...`) - math functions - an online streaming parser - minimal I/O builtions (`inputs`, `debug`) One can now write: `jq -n 'reduce inputs as $i ( ... )'` to reduce inputs in an online way without having to slurp them first! This works with streaming too. - try/catch, for catching and handling errors (this makes for a dynamic non-local exit system) - a lexical non-local exit system One can now say `label $foo | ..... | break $foo` where the break causes control to return to the label $foo, which then produces `empty` (backtracks). There's named and anonymous labels. - tail call optimization (TCO), which allows efficient recursion in jq - a variety of new control structure builtins (e.g., `while(cond; exp)`, `repeat(exp)`, `until(cond; next)`), many of which internally use TCO - an enhanced form of `reduce`: `foreach exp as $name (init_exp; update_exp; extract_exp)` - the ability to read module data files `import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar` - `--argjson var ''` Using --arg var bit me too many times :) - `--slurpfile var "filename"` Replaces the `--argfile` form (which is now deprecated but remains for backward compatibility). - support for application/json-seq (RFC7464) - a large variety of new utility functions, many being community contributions (e.g., `bsearch`, for binary searching arrays) - datetime functions - a variety of performance enhancements - `def($a): ...;` is now allowed as an equivalent of `def(a): a as $a | ...;` - test and build improvements, including gcov support Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.

jq jq-1.4

2015-08-08T05:00:03+00:00

The manual for jq 1.4 can be found at https://stedolan.github.io/jq/manual/v1.4/

jq jq-1.3

2015-08-08T05:03:24+00:00

The manual for jq 1.3 can be found at https://stedolan.github.io/jq/manual/v1.3/

jq jq-1.2

2015-08-08T05:04:02+00:00

jq 1.2

jq jq-1.0

2015-08-08T05:04:15+00:00

jq 1.0

jq jq-1.1

2015-08-08T05:04:24+00:00

jq 1.1

jq jq-1.5

2015-08-16T06:39:17+00:00

Thanks to the 20+ developers who have sent us PRs since 1.4, and the many contributors to issues and the wiki. The manual for jq 1.5 can be found at https://stedolan.github.io/jq/manual/v1.5/ Salient new features since 1.4: - regexp support (using Oniguruma)! - a proper module system `import "foo/bar" as bar; # import foo/bar.jq's defs into a bar::* namespace` and `include "foo/bar"; # import foo/bar.jq's defs into the top-level` - destructuring syntax (`. as [$first, $second, {$foo, $bar}] | ...`) - math functions - an online streaming parser - minimal I/O builtions (`inputs`, `debug`) One can now write: `jq -n 'reduce inputs as $i ( ... )'` to reduce inputs in an online way without having to slurp them first! This works with streaming too. - try/catch, for catching and handling errors (this makes for a dynamic non-local exit system) - a lexical non-local exit system One can now say `label $foo | ..... | break $foo` where the break causes control to return to the label $foo, which then produces `empty` (backtracks). There's named and anonymous labels. - tail call optimization (TCO), which allows efficient recursion in jq - a variety of new control structure builtins (e.g., `while(cond; exp)`, `repeat(exp)`, `until(cond; next)`), many of which internally use TCO - an enhanced form of `reduce`: `foreach exp as $name (init_exp; update_exp; extract_exp)` - the ability to read module data files `import "foo/bar" as $bar; # read foo/bar.json, bind to $bar::bar` - `--argjson var ''` Using --arg var bit me too many times :) - `--slurpfile var "filename"` Replaces the `--argfile` form (which is now deprecated but remains for backward compatibility). - support for application/json-seq (RFC7464) - a large variety of new utility functions, many being community contributions (e.g., `bsearch`, for binary searching arrays) - datetime functions - a variety of performance enhancements - `def($a): ...;` is now allowed as an equivalent of `def(a): a as $a | ...;` - test and build improvements, including gcov support Lastly, don't forget the wiki! The wiki has a lot of new content since 1.4, much of it contributed by the community.

jq jq-1.6

2018-11-02T01:54:23+00:00

## New in this release since 1.5: - Destructuring Alternation - New Builtins: - `builtins/0` - `stderr/0` - `halt/0, halt_error/1` - `isempty/1` - `walk/1` - `utf8bytelength/1` - `localtime/0, strflocaltime/1` - SQL-style builtins - and more! - Add support for ASAN and UBSAN - Make it easier to use jq with shebangs (8f6f28c) - Add `$ENV` builtin variable to access environment - Add `JQ_COLORS` env var for configuring the output colors ## Bug fixes: - Calling `jq` without a program argument now always assumes `.` for the program, regardless of stdin/stdout. (5fe0536) - Make sorting stable regardless of qsort. (7835a72) - Adds a local oniguruma submodule and the `./configure --with-oniguruma=builtin` option to make it easier to build with oniguruma support on systems where you can't install system-level libraries. (c6374b6 and 02bad4b) - And much more!

jq jq-1.7rc1

2023-07-31T23:04:03+00:00

Note that this is a pre-release. Feedback is welcome #2802. --- After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release. Since the last stable release many things have happened: - jq now lives at - New maintainers, admins, and owners have been recruited. - A list of [current maintainers](https://github.com/jqlang/jq/blob/70bbd10b0b58e797d03963264fc934879bb44454/AUTHORS#L4-L13) - CI, scan builds, release builds etc now use GitHub actions. @owenthereal #2596 #2620 - Lots of documentation improvements and fixes. - Web site updated with new auto complete, better section ids for linking, dark mode, etc. @itchyny #2628 - Release builds for: - Linux `amd64`, `arm64`, `armel`, `armhf`, `i386`, `mips`, `mips64`, `mips64el`, `mips64r6`, `mips64r6el`, `mipsel`, `mipsr6`, `mipsr6el`, `powerpc`, `ppc64el`, `riscv64` and `s390x` - macOS `amd64` and `arm64` - Windows `i386` and `amd64` - Docker `linux/386`, `linux/amd64`, `linux/arm64`, `linux/mips64le`, `linux/ppc64le`, `linux/riscv64` and `linux/s390x` - More details see @owenthereal #2665 - Docker images are now available from `ghcr.io/jqlang/jq` instead of docker hub. @itchyny #2652 - OSS-fuzz. @DavidKorczynski #2760 #2762 Full commit log can be found at but here are some highlights: ## CLI changes - Make object key color configurable using `JQ_COLORS` environment variable. @itchyny @haguenau @ericpruitt #2703 ```sh # this would make "field" yellow (33, the last value) $ JQ_COLORS="1;30:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 } ``` - Respect `NO_COLOR` environment variable to disable color output. See for details. @itchyny #2728 - Improved `--help` output. Now mentions all options and nicer order. @itchyny #2747 #2766 - Last output value can now control exit code using `--exit-code`/`-e`. @ryo1kato #1697 ```sh # true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4 ``` - Add `--binary`/`-b` on Windows for binary output. To get `\n` instead of `\r\n` line endings. 0dab2b1 @nicowilliams - Add `--raw-output0` for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684 ```sh # will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '%q\n' 'a b c' 'd'$'\n''e'$'\n''f' # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at ): Cannot dump a string containing NUL with --raw-output0 option ``` - Fix assert crash and validate JSON for `--jsonarg`. @wader #2658 - Remove deprecated `--argfile` option. @itchyny #2768 ## Language changes - Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752 ```sh # precision is preserved $ jq -n '100000000000000000' 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000+10' 100000000000000020 ``` - Adds new builtin `pick(stream)` to emit a projection of the input object or array. @pkoppstein #2656 ```sh $ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null } ``` - Adds new builtin `debug(msgs)` that works like `debug` but applies a filter on the input before writing to stderr. @pkoppstein #2710 ```sh $ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 } ``` - Adds new builtin `scan($re; $flags)`. Was documented but not implemented. @itchyny #1961 ```sh # look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB" ``` - Adds new builtin `abs` to get absolute value. This potentially allows the literal value of numbers to be preserved as `length` and `fabs` convert to float. @pkoppstein #2767 - Allow `if` without `else`-branch. When skipped the `else`-branch will be `.` (identity). @chancez @wader #1825 #2481 ```sh # convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3 ``` - Allow use of `$binding` as key in object literals. 8ea4a55 @nicowilliams ```sh $ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 } ``` - Allow dot between chained indexes when using `.["index"]` @nicowilliams #1168 ```sh $ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this works also $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123 ``` - Fix try/catch catches more than it should. @nicowilliams #2750 - Speed up and refactor some builtins, also remove `scalars_or_empty/0`. @muhmuhten #1845 - Now `halt` and `halt_error` exit immediately instead of continuing to the next input. @emanuele6 #2667 - Fix issue converting string to number after previous convert error. @thalman #2400 - Make 0 divided by 0 result in NaN consistently. @itchyny #2253 - Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661 - Fix deletion using assigning empty against arrays. @itchyny #2133 ```sh # now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0] ``` - Fix `stderr/0` to output raw text without any decoration. @itchyny #2751 - Fix `nth/2` to emit empty on index out of range. @itchyny #2674 - Fix `implode` to not assert and instead replace invalid unicode codepoints. @wader #2646 - Simpler and faster `transpose`. @pkoppstein #2758 - Allow keywords to be used as binding name in more places. @emanuele6 #2681 - Allow using `nan` as NaN in JSON. @emanuele6 #2712 - Fix indices/1 and rindex/1 in case of overlapping matches in strings. @emanuele6 #2718 - Enable `significand/0`, `gamma/0` and `drem/2` on macOS. @itchyny #2756 #2775 - Fix segfault when using libjq and threads. @thalman #2546

jq jq-1.7rc2

2023-08-27T18:10:22+00:00

Note that this is a pre-release. Feedback is welcome #2862. --- After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release. Since the last stable release many things have happened: - jq now lives at - New maintainers, admins, and owners have been recruited. - A list of [current maintainers](https://github.com/jqlang/jq/blob/jq-1.7/AUTHORS#L4-L14) - NEWS file is replaced by NEWS.md with Markdown format. @wader #2599 - CI, scan builds, release, website builds etc now use GitHub actions. @owenthereal @wader @itchyny #2596 #2603 #2620 #2723 - Lots of documentation improvements and fixes. - Website updated with new section search box, better section ids for linking, dark mode, etc. @itchyny #2628 - Release builds for: - Linux `amd64`, `arm64`, `armel`, `armhf`, `i386`, `mips`, `mips64`, `mips64el`, `mips64r6`, `mips64r6el`, `mipsel`, `mipsr6`, `mipsr6el`, `powerpc`, `ppc64el`, `riscv64` and `s390x` - macOS `amd64` and `arm64` - Windows `i386` and `amd64` - Docker `linux/386`, `linux/amd64`, `linux/arm64`, `linux/mips64le`, `linux/ppc64le`, `linux/riscv64` and `linux/s390x` - More details see @owenthereal #2665 - Docker images are now available from `ghcr.io/jqlang/jq` instead of Docker Hub. @itchyny #2652 #2686 - OSS-fuzz. @DavidKorczynski #2760 #2762 Full commit log can be found at but here are some highlights: ## CLI changes - Make object key color configurable using `JQ_COLORS` environment variable. @itchyny @haguenau @ericpruitt #2703 ```sh # this would make "field" bold yellow (`1;33`, the last value) $ JQ_COLORS="0;90:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 } ``` - Change the default color of null to Bright Black. @itchyny #2824 - Respect `NO_COLOR` environment variable to disable color output. See for details. @itchyny #2728 - Improved `--help` output. Now mentions all options and nicer order. @itchyny @wader #2747 #2766 #2799 - Fix multiple issues of exit code using `--exit-code`/`-e` option. @ryo1kato #1697 ```sh # true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4 ``` - Add `--binary`/`-b` on Windows for binary output. To get `\n` instead of `\r\n` line endings. @nicowilliams 0dab2b1 - Add `--raw-output0` for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684 ```sh # will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '[%s]\n' [a b c] [d e f] # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at ): Cannot dump a string containing NUL with --raw-output0 option ``` - Fix assert crash and validate JSON for `--jsonarg`. @wader #2658 - Remove deprecated `--argfile` option. @itchyny #2768 - Enable stack protection. @nicowilliams #2801 ## Language changes - Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752 ```sh # precision is preserved $ echo '100000000000000000' | jq . 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # sort/0 works $ jq -n -c '[100000000000000001, 100000000000000003, 100000000000000004, 100000000000000002] | sort' [100000000000000001,100000000000000002,100000000000000003,100000000000000004] # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000 + 10' 100000000000000020 ``` - Adds new builtin `pick(stream)` to emit a projection of the input object or array. @pkoppstein #2656 #2779 ```sh $ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null } ``` - Adds new builtin `debug(msgs)` that works like `debug` but applies a filter on the input before writing to stderr. @pkoppstein #2710 ```sh $ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 } ``` - Adds new builtin `scan($re; $flags)`. Was documented but not implemented. @itchyny #1961 ```sh # look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB" ``` - Adds new builtin `abs` to get absolute value. This potentially allows the literal value of numbers to be preserved as `length` and `fabs` convert to float. @pkoppstein #2767 - Allow `if` without `else`-branch. When skipped the `else`-branch will be `.` (identity). @chancez @wader #1825 #2481 ```sh # convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3 ``` - Allow use of `$binding` as key in object literals. @nicowilliams 8ea4a55 ```sh $ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 } ``` - Allow dot between chained indexes when using `.["index"]` @nicowilliams #1168 ```sh $ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this also works $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123 ``` - Allow dot for chained value iterator `.[]`, `.[]?` @wader #2650 ```sh $ jq -n '{"a": [123]} | .a[]' 123 # now this also works $ jq -n '{"a": [123]} | .a.[]' 123 ``` - Fix try/catch catches more than it should. @nicowilliams #2750 - Speed up and refactor some builtins, also remove `scalars_or_empty/0`. @muhmuhten #1845 - Now `halt` and `halt_error` exit immediately instead of continuing to the next input. @emanuele6 #2667 - Fix issue converting string to number after previous convert error. @thalman #2400 - Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661 - Fix deletion using assigning empty against arrays. @itchyny #2133 ```sh # now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0] ``` - Allow keywords to be used as binding name in more places. @emanuele6 #2681 - Allow using `nan` as NaN in JSON. @emanuele6 #2712 - Expose a module's function names in `modulemeta`. @mrwilson #2837 - Fix `contains/1` to handle strings with NUL. @nicowilliams 61cd6db - Fix `stderr/0` to output raw text without any decoration. @itchyny #2751 - Fix `nth/2` to emit empty on index out of range. @itchyny #2674 - Fix `implode` to not assert and instead replace invalid unicode codepoints. @wader #2646 - Fix `indices/1` and `rindex/1` in case of overlapping matches in strings. @emanuele6 #2718 - Fix `sub/3` to resolve issues involving global search-and-replace (gsub) operations. @pkoppstein #2641 - Fix `significand/0`, `gamma/0` and `drem/2` to be available on macOS. @itchyny #2756 #2775 - Fix empty regular expression matches. @itchyny #2677 - Fix overflow exception of the modulo operator. @itchyny #2629 - Fix string multiplication by 0 (and less than 1) to emit empty string. @itchyny #2142 - Fix segfault when using libjq and threads. @thalman #2546 - Fix constant folding of division and reminder with zero divisor. @itchyny #2797 - Fix `error/0`, `error/1` to throw null error. @emanuele6 #2823 - Simpler and faster `transpose`. @pkoppstein #2758 - Simple and efficient implementation of `walk/1`. @pkoppstein #2795 - Remove deprecated filters `leaf_paths`, `recurse_down`. @itchyny #2666

jq jq-1.7

2023-09-06T22:54:12+00:00

After a five year hiatus we're back with a GitHub organization, with new admins and new maintainers who have brought a great deal of energy to make a long-awaited and long-needed new release. We're very grateful for all the new owners, admins, and maintainers. Special thanks go to Owen Ou (@owenthereal) for pushing to set up a new GitHub organization for jq, Stephen Dolan (@stedolan) for transferring the jq repository to the new organization, @itchyny for doing a great deal of work to get the release done, Mattias Wadman (@wader) and Emanuele Torre (@emanuele6) for many PRs and code reviews. Many others also contributed PRs, issues, and code reviews as well, and you can find their contributions in the Git log and on the [closed issues and PRs page](https://github.com/jqlang/jq/issues?q=is%3Aclosed+sort%3Aupdated-desc). ## What's Changed Since the last stable release many things have happened: - jq now lives at - New maintainers, admins, and owners have been recruited. - A list of [current maintainers](https://github.com/jqlang/jq/blob/jq-1.7/AUTHORS#L4-L14) - NEWS file is replaced by NEWS.md with Markdown format. @wader #2599 - CI, scan builds, release, website builds etc now use GitHub actions. @owenthereal @wader @itchyny #2596 #2603 #2620 #2723 - Lots of documentation improvements and fixes. - Website updated with new section search box, better section ids for linking, dark mode, etc. @itchyny #2628 - Release builds for: - Linux `amd64`, `arm64`, `armel`, `armhf`, `i386`, `mips`, `mips64`, `mips64el`, `mips64r6`, `mips64r6el`, `mipsel`, `mipsr6`, `mipsr6el`, `powerpc`, `ppc64el`, `riscv64` and `s390x` - macOS `amd64` and `arm64` - Windows `i386` and `amd64` - Docker `linux/386`, `linux/amd64`, `linux/arm64`, `linux/mips64le`, `linux/ppc64le`, `linux/riscv64` and `linux/s390x` - More details see @owenthereal #2665 - Docker images are now available from `ghcr.io/jqlang/jq` instead of Docker Hub. @itchyny #2652 #2686 - OSS-fuzz. @DavidKorczynski #2760 #2762 Full commit log can be found at but here are some highlights: ## CLI changes - Make object key color configurable using `JQ_COLORS` environment variable. @itchyny @haguenau @ericpruitt #2703 ```sh # this would make "field" bold yellow (`1;33`, the last value) $ JQ_COLORS="0;90:0;37:0;37:0;37:0;32:1;37:1;37:1;33" ./jq -n '{field: 123}' { "field": 123 } ``` - Change the default color of null to Bright Black. @itchyny #2824 - Respect `NO_COLOR` environment variable to disable color output. See for details. @itchyny #2728 - Improved `--help` output. Now mentions all options and nicer order. @itchyny @wader #2747 #2766 #2799 - Fix multiple issues of exit code using `--exit-code`/`-e` option. @ryo1kato #1697 ```sh # true-ish last output value exits with zero $ jq -ne true ; echo $? true 0 # false-ish last output value (false and null) exits with 1 $ jq -ne false ; echo $? false 1 # no output value exists with 4 $ jq -ne empty ; echo $? 4 ``` - Add `--binary`/`-b` on Windows for binary output. To get `\n` instead of `\r\n` line endings. @nicowilliams 0dab2b1 - Add `--raw-output0` for NUL (zero byte) separated output. @asottile @pabs3 @itchyny #1990 #2235 #2684 ```sh # will output a zero byte after each output $ jq -n --raw-output0 '1,2,3' | xxd 00000000: 3100 3200 3300 1.2.3. # can be used with xargs -0 $ jq -n --raw-output0 '"a","b","c"' | xargs -0 -n1 a b c $ jq -n --raw-output0 '"a b c", "d\ne\nf"' | xargs -0 printf '[%s]\n' [a b c] [d e f] # can be used with read -d '' $ while IFS= read -r -d '' json; do > jq '.name' <<< "$json" > done < <(jq -n --raw-output0 '{name:"a b c"},{name:"d\ne\nf"}') "a b c" "d\ne\nf" # also it's an error to output a string containing a NUL when using NUL separator $ jq -n --raw-output0 '"\u0000"' jq: error (at ): Cannot dump a string containing NUL with --raw-output0 option ``` - Fix assert crash and validate JSON for `--jsonarg`. @wader #2658 - Remove deprecated `--argfile` option. @itchyny #2768 - Enable stack protection. @nicowilliams #2801 ## Language changes - Use decimal number literals to preserve precision. Comparison operations respects precision but arithmetic operations might truncate. @leonid-s-usov #1752 ```sh # precision is preserved $ echo '100000000000000000' | jq . 100000000000000000 # comparison respects precision (this is false in JavaScript) $ jq -n '100000000000000000 < 100000000000000001' true # sort/0 works $ jq -n -c '[100000000000000001, 100000000000000003, 100000000000000004, 100000000000000002] | sort' [100000000000000001,100000000000000002,100000000000000003,100000000000000004] # arithmetic operations might truncate (same as JavaScript) $ jq -n '100000000000000000 + 10' 100000000000000020 ``` - Adds new builtin `pick(stream)` to emit a projection of the input object or array. @pkoppstein #2656 #2779 ```sh $ jq -n '{"a": 1, "b": {"c": 2, "d": 3}, "e": 4} | pick(.a, .b.c, .x)' { "a": 1, "b": { "c": 2 }, "x": null } ``` - Adds new builtin `debug(msgs)` that works like `debug` but applies a filter on the input before writing to stderr. @pkoppstein #2710 ```sh $ jq -n '1 as $x | 2 | debug("Entering function foo with $x == \($x)", .) | (.+1)' ["DEBUG:","Entering function foo with $x == 1"] ["DEBUG:",2] 3 $ jq -n '{a: 1, b: 2, c: 3} | debug({a, b, sum: (.a+.b)})' ["DEBUG:",{"a":1,"b":2,"sum":3}] { "a": 1, "b": 2, "c": 3 } ``` - Adds new builtin `scan($re; $flags)`. Was documented but not implemented. @itchyny #1961 ```sh # look for pattern "ab" in "abAB" ignoring casing $ jq -n '"abAB" | scan("ab"; "i")' "ab" "AB" ``` - Adds new builtin `abs` to get absolute value. This potentially allows the literal value of numbers to be preserved as `length` and `fabs` convert to float. @pkoppstein #2767 - Allow `if` without `else`-branch. When skipped the `else`-branch will be `.` (identity). @chancez @wader #1825 #2481 ```sh # convert 1 to "one" otherwise keep as is $ jq -n '1,2 | if . == 1 then "one" end' "one" 2 # behaves the same as $ jq -n '1,2 | if . == 1 then "one" else . end' "one" 2 # also works with elif $ jq -n '1,2,3 | if . == 1 then "one" elif . == 2 then "two" end "one" "two" 3 ``` - Allow use of `$binding` as key in object literals. @nicowilliams 8ea4a55 ```sh $ jq -n '"a" as $key | {$key: 123}' { "a": 123 } # previously parentheses were needed $ jq -n '"a" as $key | {($key): 123}' { "a": 123 } ``` - Allow dot between chained indexes when using `.["index"]` @nicowilliams #1168 ```sh $ jq -n '{"a": {"b": 123}} | .a["b"]' 123 # now this also works $ jq -n '{"a": {"b": 123}} | .a.["b"]' 123 ``` - Allow dot for chained value iterator `.[]`, `.[]?` @wader #2650 ```sh $ jq -n '{"a": [123]} | .a[]' 123 # now this also works $ jq -n '{"a": [123]} | .a.[]' 123 ``` - Fix try/catch catches more than it should. @nicowilliams #2750 - Speed up and refactor some builtins, also remove `scalars_or_empty/0`. @muhmuhten #1845 - Now `halt` and `halt_error` exit immediately instead of continuing to the next input. @emanuele6 #2667 - Fix issue converting string to number after previous convert error. @thalman #2400 - Fix issue representing large numbers on some platforms causing invalid JSON output. @itchyny #2661 - Fix deletion using assigning empty against arrays. @itchyny #2133 ```sh # now this works as expected, filter out all values over 2 by assigning empty $ jq -c '(.[] | select(. >= 2)) |= empty' <<< '[1,5,3,0,7]' [1,0] ``` - Allow keywords to be used as binding name in more places. @emanuele6 #2681 - Allow using `nan` as NaN in JSON. @emanuele6 #2712 - Expose a module's function names in `modulemeta`. @mrwilson #2837 - Fix `contains/1` to handle strings with NUL. @nicowilliams 61cd6db - Fix `stderr/0` to output raw text without any decoration. @itchyny #2751 - Fix `nth/2` to emit empty on index out of range. @itchyny #2674 - Fix `implode` to not assert and instead replace invalid unicode codepoints. @wader #2646 - Fix `indices/1` and `rindex/1` in case of overlapping matches in strings. @emanuele6 #2718 - Fix `sub/3` to resolve issues involving global search-and-replace (gsub) operations. @pkoppstein #2641 - Fix `significand/0`, `gamma/0` and `drem/2` to be available on macOS. @itchyny #2756 #2775 - Fix empty regular expression matches. @itchyny #2677 - Fix overflow exception of the modulo operator. @itchyny #2629 - Fix string multiplication by 0 (and less than 1) to emit empty string. @itchyny #2142 - Fix segfault when using libjq and threads. @thalman #2546 - Fix constant folding of division and reminder with zero divisor. @itchyny #2797 - Fix `error/0`, `error/1` to throw null error. @emanuele6 #2823 - Simpler and faster `transpose`. @pkoppstein #2758 - Simple and efficient implementation of `walk/1`. @pkoppstein #2795 - Remove deprecated filters `leaf_paths`, `recurse_down`. @itchyny #2666 **Full Changelog**: https://github.com/jqlang/jq/compare/jq-1.7rc2...jq-1.7

jq jq-1.7.1

2023-12-13T19:56:17+00:00

## Security - CVE-2023-50246: Fix heap buffer overflow in jvp\_literal\_number\_literal - CVE-2023-50268: fix stack-buffer-overflow if comparing nan with payload ## CLI changes - Make the default background color more suitable for bright backgrounds. @mjarosie @taoky @nicowilliams @itchyny #2904 - Allow passing the inline jq script after `--`. @emanuele6 #2919 - Restrict systems operations on OpenBSD and remove unused `mkstemp`. @klemensn #2934 - Fix possible uninitialised value dereference if `jq_init()` fails. @emanuele6 @nicowilliams #2935 ## Language changes - Simplify `paths/0` and `paths/1`. @asheiduk @emanuele6 #2946 - Reject `U+001F` in string literals. @torsten-schenk @itchyny @wader #2911 - Remove unused nref accumulator in `block_bind_library`. @emanuele6 #2914 - Remove a bunch of unused variables, and useless assignments. @emanuele6 #2914 - main.c: Remove unused EXIT\_STATUS\_EXACT option. @emanuele6 #2915 - Actually use the number correctly casted from double to int as index. @emanuele6 #2916 - src/builtin.c: remove unnecessary jv\_copy-s in type\_error/type\_error2. @emanuele6 #2937 - Remove undefined behavior caught by LLVM 10 UBSAN. @Gaelan @emanuele6 #2926 - Convert decnum to binary64 (double) instead of decimal64. This makes jq behave like the JSON specification suggests and more similar to other languages. @wader @leonid-s-usov #2949 - Fix memory leaks on invalid input for `ltrimstr/1` and `rtrimstr/1`. @emanuele6 #2977 - Fix memory leak on failed get for `setpath/2`. @emanuele6 #2970 - Fix nan from json parsing also for nans with payload that start with 'n'. @emanuele6 #2985 - Allow carriage return characters in comments. @emanuele6 #2942 #2984 ## Documentation changes - Generate links in the man page. @emanuele6 #2931 - Standardize arch types to AMD64 & ARM64 from index page download dropdown. @owenthereal #2884 ## libjq - Add extern C for C++. @rockwotj #2953 ## Build and test changes - Fix incorrect syntax for checksum file. @kamontat @wader #2899 - Remove `-dirty` version suffix for windows release build. @itchyny #2888 - Make use of `od` in tests more compatible. @nabijaczleweli @emanuele6 @nicowilliams #2922 - Add dependabot. @yeikel #2889 - Extend fuzzing setup to fuzz parser and and JSON serializer. @DavidKorczynski @emanuele6 #2952 - Keep releasing executables with legacy names. @itchyny #2951 ## New Contributors * @yeikel made their first contribution in https://github.com/jqlang/jq/pull/2889 * @dependabot made their first contribution in https://github.com/jqlang/jq/pull/2894 * @kamontat made their first contribution in https://github.com/jqlang/jq/pull/2899 * @taoky made their first contribution in https://github.com/jqlang/jq/pull/2904 * @tboz203 made their first contribution in https://github.com/jqlang/jq/pull/2920 * @nabijaczleweli made their first contribution in https://github.com/jqlang/jq/pull/2922 * @klemensn made their first contribution in https://github.com/jqlang/jq/pull/2934 * @asheiduk made their first contribution in https://github.com/jqlang/jq/pull/2946 * @rockwotj made their first contribution in https://github.com/jqlang/jq/pull/2953 * @jesperronn made their first contribution in https://github.com/jqlang/jq/pull/2898 **Full Changelog**: https://github.com/jqlang/jq/compare/jq-1.7...jq-1.7.1