Compatibility and Compliance

Jawk is checked during mvn verify against 3 upstream compatibility families: POSIX awk specifications[1], One True Awk[2], GNU Awk[3]. This page summarizes what currently passes, fails, errors, or remains intentionally skipped.

POSIX

Specification-focused coverage transcribed from the POSIX awk utility requirements[1].

95 / 95
Passed Failed Error Skipped
100%

One True Awk (BWK)

Compatibility coverage derived from the original One True Awk[2] test collection.

221 / 225
Passed Failed Error Skipped
98%

GNU Awk (gawk)

GNU Awk[3] compatibility gathered from explicit Java integration suites that mirror the main gawk test families: core behavior, extensions, locale-sensitive behavior, and optional features.

261 / 624
Passed Failed Error Skipped
41%

Detailed Behavior Notes

Date and time functions

The date and time functions (mktime(), strftime()) follow the Java platform's time zone data and calendar rules rather than the C library's, which differs from gawk in a few edge cases:

  • ENVIRON["TZ"] accepts the zone IDs Java understands: Olson names such as Europe/Paris (optionally with a POSIX : prefix) and Java's custom GMT+3 IDs with Java's sign convention (GMT+3 is UTC+03:00, where POSIX would read UTC-03:00). POSIX TZ rule specifications such as XXX3 or CET-1CEST,M3.5.0,M10.5.0/3 are not parsed; unknown zones fall back to GMT. An explicitly empty TZ selects UTC, as in POSIX.
  • An ambiguous wall time during a DST fall-back resolves to its standard-time occurrence; the explicit DST field of mktime() still selects either occurrence.
  • A positive mktime() DST hint applies the zone's current daylight adjustment; zones without daylight saving time ignore the hint.
  • strftime()'s %Z prints the zone's current designation (the JDK's time zone data records historical offsets and DST rules, but not historical zone names), and timestamps before the common era use the JDK's year numbering rather than astronomical (negative) years.

Integer arithmetic

Arithmetic on integral operands is computed in exact 64-bit integers when both the operands and the result fit in 64 bits: +, -, *, an evenly-dividing /, %, unary - and +, ++/--, and the corresponding compound assignments, on variables, array elements, and fields alike. Comparisons between two such values are exact as well.

Within ±2^53 this is indistinguishable from gawk. Beyond that, Jawk keeps every digit where gawk — which computes in C doubles unless built with MPFR support and run with -M — rounds: print 9007199254740992 + 1 prints 9007199254740993 in Jawk and 9007199254740992 in default gawk. A result that overflows 64 bits, an exponentiation (^), and any operation with a fractional or string operand fall back to IEEE double arithmetic, matching gawk exactly.

Two consequences are worth knowing:

  • A compound assignment or ++/-- on an uninitialized variable, a missing array element, or a missing field starts from integer zero, so counters built that way stay exact. A field assigned or updated with a numeric value retains the number itself: print $1 shows the full value, repeated updates stay exact, and the value is converted with CONVFMT only when $0 is reconstituted, as in gawk.
  • The integer domain has no negative zero. When x is integer zero, -x yields plain 0 where double arithmetic would keep -0.0, so an expression that can observe the sign, such as 1/-x, prints inf. (gawk treats any division by zero as a fatal error, which Jawk deliberately does not.)

printf and sprintf formatting

printf and sprintf implement the POSIX AWK conversions with gawk's semantics: %s converts numbers with CONVFMT (integral values print without a fractional part), %c prints the character of a numeric code point or the first character of a string, %i is an alias for %d, dynamic * width and precision consume arguments (a negative width left-justifies, a negative precision means no precision), gawk positional specifiers (%2$s) and the ' grouping flag are honored, out-of-range integers wrap or fall back exactly as in gawk, halfway cases round to even (printf "%.0f", 2.5 prints 2), too few arguments is a fatal error, and unknown conversion specifiers print verbatim without consuming an argument.

A few edge cases follow Java's platform rules rather than the C library's:

  • %c is locale-independent: a numeric argument selects the Unicode code point (so printf "%c", 233 always prints é), where C-locale gawk emits the raw byte. Values that are not valid code points are truncated to a UTF-16 char. Field widths and %s precision count characters (code points) regardless of locale, like gawk in a multibyte locale — except that gawk pads %c by bytes, which Jawk does not reproduce.
  • %a/%A use Java's hexadecimal floating-point notation (0x1.0p0 where glibc prints 0x1p+0); gawk itself documents these conversions as C-library dependent.
  • NaN always prints as nan: Java does not track the sign of NaN, so gawk's occasional -nan is rendered without a sign.

Special filenames

Jawk recognizes the gawk special filenames[7] in redirections and in getline, and routes them to the streams the process already holds open instead of opening files of those names:

  • /dev/stdout and /dev/fd/1 in print/printf redirections write to the standard output, interleaved with unredirected print output.
  • /dev/stderr and /dev/fd/2 write to the standard error, interleaved with the diagnostics Jawk itself writes there. Every record is flushed, as gawk keeps /dev/stderr unbuffered.
  • /dev/stdin and /dev/fd/0 in getline < file read the standard input, exactly like the conventional - operand does.

Nothing is opened or truncated, so > and >> behave identically on these names, and a shell redirection such as 2>log keeps its position: the records the script writes and the ones Jawk writes go through the same stream instead of clobbering each other.

close() on one of these names flushes the redirection and reports success, but never closes the underlying descriptor — which is shared with the runtime and, when Jawk is embedded, with the host application — so the name stays usable afterwards. As in gawk, closing a name no redirection is open on returns -1, and the records a closed /dev/stdin had already buffered are gone with it.

Other /dev/fd/N names are not special-cased and are opened as ordinary files, which works on the platforms that provide them (as gawk itself documents for systems without /dev/fd support). These names are only special in redirections and in getline: an operand naming an input file is opened as a regular file, so use - to read the standard input there. In sandbox mode[8] all redirections are rejected, the special filenames included.

/dev/null is not one of those four names but a real device on every Unix system, so portable scripts discard output by redirecting to it. Jawk makes the name mean the same thing on Windows, where the device is spelled NUL, as gawk's Windows port does: print > "/dev/null" and printf > "/dev/null" discard their output and create no file, getline < "/dev/null" reports end of input (0), close("/dev/null") succeeds, and /dev/null as an operand reads as an empty input file. Unlike the four names above, the null device is opened like any other file — only its name is translated, never the redirection itself — so close() takes the spelling the script used. The native Windows spelling NUL needs no translation, since the file system opens that name as the device itself, and it is accepted wherever /dev/null is, operands included.

Reading input with getline

Each getline variant sets exactly the variables gawk documents for it:

Variant Sets
getline $0, NF, NR, FNR
getline var var, NR, FNR
getline < file $0, NF
getline var < file var
cmd \| getline $0, NF
cmd \| getline var var

In particular, no redirected form touches NR, FNR, or FILENAME. For the pipe forms POSIX prescribes updating NR, but gawk documents and implements leaving it alone, and Jawk follows gawk.

Every variant returns 1 when a record was read, 0 at end of input, and the redirected forms return -1 when the file cannot be opened or the command cannot be spawned, with ERRNO carrying the gawk-style description (No such file or directory, Is a directory, …), so a script can probe for optional files with the idiomatic while ((getline line < f) > 0) loop. A failed open is not cached: once the file exists, the same getline succeeds. A redirection name that evaluates to the empty string is a fatal error, as in gawk (expression for<' redirection has null string value), not a-1. Whengetlinereturns0or-1it has read nothing and sets nothing:$0, its fields, and the target variable keep their previous values. One dark corner follows gawk 4.0 and later (theirgetline5test): with an array or field target, the subscript expression is still evaluated when nothing is read —getline a[++c] < fadvancescon every attempt, the reference creates the array element, and an invalid field target such asgetline $(-1) < f` is rejected — but nothing is stored. The one detail Jawk does not reproduce is gawk's exact timing: gawk computes the subscript before attempting the read, Jawk after, which is only observable when the subscript expression itself inspects or assigns ERRNO.

Range patterns

Range patterns (begpat, endpat) evaluate their two conditions lazily, as POSIX requires: the start condition is evaluated only while outside the range, and the end condition only once the range has started — including on the very record that starts it, so a range can begin and end on the same record. Conditions with side effects, such as a++ == 2, a++ == 5, therefore behave exactly as in gawk and One True Awk: each condition's side effects run only on the records where that condition is actually tested.

BEGINFILE and ENDFILE

BEGINFILE rules run just before the first record of each input file is read, with FILENAME set, FNR at 0, $0 cleared, and ARGIND designating the ARGV entry being processed. ENDFILE rules run when the last record of a file has been consumed — even for empty files — and before the END rules for the last file. With a BEGINFILE rule present, a file that cannot be opened is not an immediate fatal error: ERRNO carries the description and the rule can skip the file with nextfile, as shown in the CLI guide[9].

nextfile may also be used in ordinary rules, including inside user-defined functions, and abandons the rest of the current input file after running the ENDFILE rules. next may likewise be used inside a user-defined function: it abandons the current record and unwinds the active function calls, and is a fatal error at runtime when the function was called from a BEGIN, END, BEGINFILE, or ENDFILE rule. Direct uses in special rules are rejected at compile time: next inside BEGIN, END, BEGINFILE, or ENDFILE rules, and nextfile inside ENDFILE, BEGIN, and END rules. Only redirected forms of getline (such as getline line < "file") may be used inside BEGINFILE/ENDFILE. All of this matches gawk's restrictions.

When BEGINFILE/ENDFILE rules are present, a non-redirected getline in an ordinary rule never crosses a file boundary: it reports end-of-input at the end of the current file, and the main loop then runs the ENDFILE and BEGINFILE rules before the next file's records are processed. (gawk instead lets such a getline pull in the next file's first record, firing the hooks mid-statement.) Without BEGINFILE/ENDFILE rules, getline keeps the classic AWK behavior of streaming across input files.

As in gawk, BEGINFILE and ENDFILE are gawk extensions: with --posix they are not special and parse as ordinary identifiers.

gawk source syntax

@include resolves relative paths from the including source, then searches AWKPATH, and includes each resolved file at most once. An included file cannot include a top-level program source. An included source begins in the awk namespace; the including source's namespace is restored afterward.

@namespace qualifies variables and functions except identifiers made entirely of uppercase letters, while awk::name refers to the default namespace. An indirect call (@selector(args)) can dispatch a user-defined, built-in, or loaded extension function; in a namespaced program the selector variable must hold a fully qualified name, since an unqualified one refers to the default awk namespace.

@load is recognized but intentionally reported as unsupported: load Java extensions[10] with the CLI -l option or the Java API instead.

All gawk @ forms — typed regexp literals (@/re/) included — are rejected in POSIX mode.

SYMTAB and FUNCTAB

Scripts that reference SYMTAB or FUNCTAB get honest, Jawk-shaped content, populated by the runtime itself (outside POSIX mode): SYMTAB holds the names of the program's globals, Jawk's special variables, and -v/host-supplied variables; FUNCTAB holds the names of the standard built-in functions (split, substr, …), the program's user-defined functions, and the loaded extensions' function keywords. Reads and writes through SYMTAB reflect declared globals and managed special variables live, but arbitrary elements cannot be added or deleted and writes must preserve each global's scalar or array type. FUNCTAB is read-only. As in gawk, assigning a scalar to SYMTAB or FUNCTAB is a runtime error.

Where Jawk follows gawk

Jawk follows gawk in several places where gawk departs from historical AWK:

  • A regexp literal used as an ordinary expression evaluates as $0 ~ /re/, per POSIX; x = /re/ assigns 0 or 1, not the pattern text.
  • Typed regexp literals (@/re/) are accepted; --posix rejects them.
  • IGNORECASE is a built-in special variable managed by the runtime, like FS or RS: a truthy value makes regexp matching, field and split() separators, string comparisons, index(), and gawk array sorting case-insensitive.
  • Because the gawk compatibility extension[11] is enabled by default, gensub, typeof, isarray, asort, asorti, mkbool, patsplit, strtonum, systime, mktime, strftime, bindtextdomain, dcgettext, and dcngettext are function names and can no longer be used as variables. Load an explicit extension list without GawkExtension to reclaim those identifiers.
  • Calling a user-defined function with more arguments than it declares is accepted: the extra expressions are evaluated and discarded, and a gawk-style warning is printed to stderr.
  • Array and scalar types follow gawk's runtime rules across function parameters and nested array references. An untyped argument remains linked to its caller until its first scalar or array use fixes the type; conflicting assignment, membership, iteration, deletion, or subarray use then fails at runtime.
  • Assigning a scalar value to an array element (a[1] = z) types that element as a scalar; later using it as a subarray (a[1][2] = 3) is a runtime error, exactly as in gawk.
  • Integral values too large for a 64-bit integer print with all their digits, as in gawk: print 1e20 outputs 100000000000000000000 and print 2^100 outputs 1267650600228229401496703205376, rather than falling back to OFMT exponential notation. An e/E in a numeric constant that is not followed by a valid exponent is not part of the number, as in gawk: 1e is the number 1 followed by the variable e.
  • print and printf resolve the leading-parenthesis ambiguity as gawk does: print (a, b) prints the parenthesized group as the whole argument list (which is how print (a, b) > "file" disambiguates a redirection from a comparison), while a single parenthesized expression followed by a comma continues the output list, as in print (i==0), (i==""), and a group followed by in is a membership key, as in print (1,2) in a. A comma group followed by more arguments, such as print (1,2), 3, is a syntax error. More generally, outside of a print/printf argument list, a parenthesized expression list such as (i, j) is a grouping, not an expression, and is only valid immediately before in, as in ((i, j) in array).

See Also

compatibility differences awk bwk mawk gawk jawk awk
Links:
  • [1] https://pubs.opengroup.org/onlinepubs/9699919799/utilities/awk.html
  • [2] https://github.com/onetrueawk/awk
  • [3] https://www.gnu.org/software/gawk/
  • [4] failsafe.html#io-jawk-posix
  • [5] failsafe.html#io-jawk-onetrueawk
  • [6] failsafe.html#io-jawk-gawk
  • [7] https://www.gnu.org/software/gawk/manual/html_node/Special-FD.html
  • [8] cli.html#enable-sandbox-mode
  • [9] cli.html#beginfile-and-endfile-rules
  • [10] extensions.html
  • [11] extensions.html#gawk
  • [12] behavior-changes.html
  • [13] cli-reference.html
  • [14] java-compile.html
Searching...
No results.