mkiocccentry 1.1 2024-05-19
Release 1.1 2024-05-19
Release version 1.1 of the mkiocccentry tool set.
Release 1.0.64 2024-05-18
Changed all submission_num to submit_slot (including json tags) and changed
all MAX_SUBMISSION_NUM to MAX_SUBMIT_SLOT_NUM and
all MAX_SUBMISSION_CHARS to MAX_SUBMIT_SLOT_CHARS and
changed relater comment and prompt text to be more consistent with the
pending submit server AND reduce the chance that someone revising their
submission incorrectly thinks they need to increment that value each time.
Remove formed_UTC JSON member from .auth.json and .info.json.
The formed_UTC JSON member was redundant in that it was simply
a different coding of the formed_timestamp in ASCII date string form.
It was a source of errors in editing formed_timestamp values.
Changed INFO_VERSION from "1.15 2024-03-02"
to "1.16 2024-05-18" as a result of the JSON member name above changes.
Changed AUTH_VERSION from "1.21 2024-03-04"
to "1.22 2024-05-18" s a result of the JSON member name above changes.
Fix soup/reset_tstamp.sh and soup/Makefile to look for the
correct location of the limit_ioccc.h file and fixed the prompts
result messages.
Changed soup/reset_tstamp.sh RESET_TSTAMP_VERSION from "1.0.1 2024-03-02"
to "1.0.2 2024-05-15" as a result of the above changes.
Changed MKIOCCCENTRY_VERSION from "1.0.5 2024-05-15"
to "1.0.6 2024-05-19" as a result of the above changes.
Changed TXZCHK_VERSION from "1.0.4 2024-05-16"
to "1.0.5 2024-05-19" as a result of the above changes.
Performed make reset_min_timestamp to change MIN_TIMESTAMP from 1675547786
to 1716103155 due to the changes in both .auth.json and .info.json.
Release 1.0.63 2024-05-16
Changed more references of entry to submission and entries to
submissions and updated the man pages for mkiocccentry(1) and chkentry(1)
about this as there are places where it is NOT appropriate to update the terms.
It might be that there are no more places where entry or entries is used
incorrectly (except perhaps for iocccsize.c) but the other terms have not been
checked.
Make txzchk(1) not show output if -q specified. Updated version to "1.0.4 2024-05-16".
The mkiocccentry(1) tool prints both the location name and the
common location name when prompting the user to validate author
location data.
Release 1.0.62 2024-05-15
Changed MKIOCCCENTRY_REPO_VERSION from "1.0.1 2023-02-14"
to "1.0.62 2024-05-15" to match the above release version.
Changed IOCCC_CONTEST from "IOCCCMOCK"
to "IOCCC28" for testing purposes. :-)
Changed IOCCC_YEAR from "2023"
to "2024" for testing purposes. :-)
Changed MKIOCCCENTRY_VERSION from "1.0.4 2024-03-04"
to "1.0.5 2024-05-15" because of the above constant changes.
The filename formed by mkiocccentry(1) was changed
to start with submit. instead of entry..
Changed FNAMCHK_VERSION from "1.0.1 2024-03-02"
to "1.0.2 2024-05-15" because of the above change.
Updated mkiocccentry(1) and chkentry(1) man pages accordingly.
Updated the test files according to the above changes.
Release 1.0.61 2024-03-07
Remove from comments the winner_handle. Fixed the regexp of an allowed handle
(in comments and in strings). It is regrettable but due to the use of
posix_plus_safe() additional characters are allowed that might be better left
out (these have been allowed for a long time but the comments and strings said
otherwise). The regexp that is allowed for author_handles is:
^[0-9A-Za-z][0-9A-Za-z._+-]*$The difference is that it allows upper case characters too. In particular the
old regexp was ^[0-9a-z][0-9a-z._+-]*$.
Typo fix in txzchk.h: referred to posix_safe_plus() instead of
posix_plus_safe().
Change words entry and entries to submission and submissions (exception:
in iocccsize).
Added FAQ 12 "Why do these tools sometimes use the incorrect IOCCC terms?".
Release 1.0.59 2024-03-04
Added a distinction between a "common name" and "name" for locations.
Added -c to the location command line to change using "name"
(the default) to ""common name".
Removed location_name JSON member from .auth.json.
While mkiocccentry will ask the user to verify their location_code
by also showing them the location_name, it will no longer add
the JSON member with the JSON member name of "location_name" to .auth.json.
Changed LOCATION_VERSION from "1.0.2 2023-08-04" to "1.0.3 2024-03-03".
Changed MKIOCCCENTRY_VERSION from "1.0.3 2024-03-02 to "1.0.4 2024-03-04".
Changed IOCCC_auth_version from "1.20 2024-03-02" to "1.21 2024-03-04".
Changed MKIOCCCENTRY_REPO_VERSION from "1.0.1 2023-02-14" to "1.1 2023-03-04".
Added a simple make unstall rule to uninstall all files installed by this repo.
Release 1.0.58 2024-03-02
Address part (possibly all) of the issue of new IOCCC terms (only part because
some things need to be decided still especially with file names and related
specifics but possibly all if it's decided these should stay the same).
Changed version of txzchk to "1.0.3 2024-03-02".
Changed version of fnamchk to "1.0.1 2024-03-02".
Changed version of mkiocccentry to "1.0.3 2024-03-04".
Changed AUTH_VERSION to "1.20 2024-03-02".
Changed INFO_VERSION to "1.15 2024-03-02".
In many cases the word entry was changed to submission. For chkentry and
mkiocccentry it has been decided to keep the names. However in text, variable
and comments the word is now submission. It is not clear, however, if a
submission directory needs to have a different prefix (currently it's entry.).
If this is necessary then more changes will have to be made. If it needs to be
renamed then changes in fnamchk and txzchk plus data files (at the very
least - there might be other places that'd have to be updated) will have to be
made.
winner/winners changed to winning author/winning authors (a more
thorough check has to be performed to make sure all are caught, see below). A
winning submission is an entry and at least in some cases this is now clear. In
the case of winner_handle (or even just winner handle) this was not changed.
There already exists an author_handle and that's what is in the files but in
comments winner_handle also exists. In other words this ambiguity should be
addressed.
The json past_winner was changed to past_winning_author. This
involved updating some patch files and all the auth json files as well.
Typo fix in bug_report.sh. No version update.
In many cases (not all) the year 2023 was changed to 2024. The IOCCC_YEAR was
NOT updated to 2024 as this causes other problems that require a bit more time.
A fun problem to resolve is the text along the lines of 'submit your entry'
being changed to 'submit your submission'. In some cases the surrounding text
was reworded but not in all. If you like this can be submissions all the way
down!
Release 1.0.57 2024-01-21
Rename "print_test" to "pr_jparse_test". The name "print_test" was
too generic and might conflict with some test print application
if it was installed.
Release 1.0.56 2024-01-20
Changed "iocccsize_version" in JSON files (those that were "28.13 2023-02-04") to ""28.14 2024-01-04". This fixed a problem with make prep
failing due to the script chkentry_test.sh failing due to many errors
reported, some of which didn't seem to make any sense whatever.
Change fd_is_ready() to return true if isatty(fd).
Release 1.0.55 2023-11-07
Add another forbidden file name in submissions: prog.alt (prog.alt.c is
allowed).
Modularise the checks for invalid filenames in entries. For instance in
check_extra_file() there's no need to check each extra filename and then give
the same error message changing the macro of the filename that's disallowed when
we can just print the string being tested against. The only difference is that
there's one if (with multiple checks) and instead of duplicating the same error
message we just print it once with the string being tested against. Note that
there are two sets of checks: one for extra files being required filenames and
another for disallowed filenames.
Make sure to use the macros for the filenames, not the literal strings (e.g. use
PROG_FILENAME not "prog").
Check filenames in alphabetical order (I think :-) .. very tired so maybe missed
one or two).
Note that the function check_extra_file() CANNOT be used in every case so it's
not used except in chkentry!
Release 1.0.54 2023-11-06
The following filenames are no longer allowed in an entry's extra files list:
- README.md
- prog
- prog.orig
- prog.orig.c
- index.html
- inventory.html
The mkiocccentry tool checks for this, giving an error early on (rather than
making the user retry after inputting more details). The function
test_extra_file() now checks for these files too, giving an error if any are
found.
New mkiocccentry version: 1.0.2 2023-11-06.
txzchk also checks for these files in the tarball. New version: 1.0.2
2023-11-06.
Release 1.0.53 2023-09-13
Minor fix in jval and jnamval - new version "0.0.15 2023-09-13". The ne
operator for -S and -n already was parsed (by foresight that it would
be needed, maybe) but it was not in the help. Also I decided to change the order
of the enum so that JSON_CMP_OP_NE comes right after JSON_CMP_OP_EQ rather
than at the end.
Man pages updated - added ne to -S and -n options.
Release 1.0.52 2023-08-09
New version of jval and jnamval, "0.0.14 2023-08-09".
Fixed the lists of -S and -n option parsing. For now it should just be a
string that's strdup()d. The operators are in the list in the order specified,
one list for strings and one list for numbers. This can be changed to a single
list later on if necessary (as I suspect it might need to be but the way it is
now is set up as two which is what I'm operating under).
The function that frees the -S and -n lists in jval and jnamval is now
in json_util.c as they are actually in a struct common to both jval and
jnamval. The functions that free them in jval_util.c and jnamval_util.c
simply check that jval or jnamval is not NULL and then calls the new
free_json_util_cmp_list() function.
Made the json util operator macros an enum.
Rename the enum output_format to JSON_UTIL_OUTPUT_FMT.
Release 1.0.51 2023-08-07
Fix link in make rule jparse.clone to use
https://github.com/xexyl/jparse.git as the link used,
https://github.com/lcn2/jparse.git, is actually a clone of the repo I set up
(empty for now) which will be populated with jparse later on when the parser
is tested more fully.
Add -F to common options of json_util_README.md.
Change jnamval -P to jnamval -r (restrict printing types). There might be a
reference to something non-programming in this change. New version "0.0.13 2023-08-07".
Slightly improve dbg_example.c by adding a call to not_reached() after the
errp() call.
Release 1.0.50 2023-08-06
New version of jfmt, jval and jnamval: "0.0.12 2023-08-06". jfmt and
jval jumped from 0.0.10 to make them match. Later on after version 1.0.0 this
will no longer happen.
Updated the -F option list. It now is:
-f fmt change the json format style (def: use default)
tty when output is to a tty, use colour, otherwise use simple
simple each line has one json level determined by '[]'s and '{}'s
colour simple plus ansi colour syntax highlighting
color alias for colour (colour excluding you :-) )
1line one line output
nows One line output, no extra whitespace
Updated man pages for the three tools for this change as well as a typo fix in
jnamval.1 for the -p option (wrongly listed as -P which is something
else).
Updated tmp/TODO.md: the options for the three tools should be in the same
order, particularly the common options, both in usage string and man pages. But
since the specs are not finalised this needn't be done yet.
Release 1.0.49 2023-08-05
New version of jnamval: "0.0.11 2023-08-05". Add -I option with same
semantics as jfmt. Updated jnamval.1 to add this option and fix typos.
Change available options for jnamval -t to be that of jval -t. Updated man
page.
Typo fixes in jfmt.1 and jval.1. Removed -I from jval.1 as it's not a
valid option.
Fix typo in usage string in jnamval which referred to non-existent option (it
is actually an option arg, not an option, that is correct and that is now what
it says).
Add option -P to jnamval. This is just like what -t for jnamval used to
be but it will indicate what to print rather than what to match. Man page
updated for this option. As the code is common to both jval and jnamval the
jval utility could be updated easily to support this option is desired but
this is TBD later.
Add some bad test JSON files in the jparse/test_jparse/test_JSON/bad_loc
directory, updated Makefile rule Added appropriate err files.
Updated Makefile rule rebuild_jparse_err_files to not show the command being
run for each file to make the output clean (and to show what it looks like in
the README.md file - which has been updated due to slight changes).
Fix warning message of mkiocccentry -y and move it to happen before the option
to ignore all warnings as that one says no more warnings will be displayed so it
should come last.
Removed the "program" arg from parse_verbosity(). The parse_verbosity()
no longer exits on errors or bad arguments. Instead parse_verbosity()
returns DBG_INVALID, a value < 0.
The return from parse_verbosity() is now checked for a < 0 return
value and an appropriate error is raised when that happens.
Release 1.0.48 2023-08-04
Bug fixes in location tool wrt the -a and -s options. New version "1.0.2 2023-08-04".
Change -N to -n.
Always use re-entrant version of the lookup functions as this allows for proper
showing of both name and code whether or not one is using -s or -n. If -a
we still show all (as before) but without -n we previously showed both name
and code like code ==> name when it should only show both if verbosity level
is > 0. Note that the chkentry tool SHOULD NOT and DOES NOT use the
re-entrant versions.
Use parse_verbosity() for parsing the -v option.
Updated man page with an example added and updating -N to -n.
Release 1.0.47 2023-08-03
New version of location tool: "1.0.1 2023-08-03".
Improved tool to allow for -a (show all) and -s (substring search). Uses new
re-entrant versions of the functions lookup_location_name() and
lookup_location_code(). These new functions take a size_t *idx which is set
to the next element in the table (if != NULL) so that the next call can skip
continue after the previous element. They also take a struct location **
which stores the found location. This is useful for -v 1 as one can then see
something like:
./location -a -s -N -v 1 'united'
United Arab Emirates ==> AE
United Kingdom of Great Britain and Northern Ireland (the) ==> GB
United States Miscellaneous Pacific Islands ==> PU
Tanzania, United Republic of ==> TZ
United Kingdom ==> UK
United States Minor Outlying Islands ==> UM
United Nations ==> UN
United States of America ==> USWithout the -v 1 it will only show:
$ ./location -a -s -N 'united'
AE
GB
PU
TZ
UK
UM
UN
USIf one does not use the -N option the -a is less useful as that function
checks explicitly that the length is two characters so it has to be an exact
match. Nevertheless there is a re-entrant version of the function that works
much the same way as the other and the -a is processed without -N. Use of
-s requires an arg much like -N but it does NOT require -N itself.
The rationale behind these changes is they will make it easier for people to
find their country code, if they do not know what it is (or they want say
anonymous and don't know that it's XX). Search is done case-insensitively.
Another example use:
$ ./location -asNv 1 germ
German Democratic Republic ==> DD
Germany ==> DEI, that is Cody/@xexyl, observe that Germany is a country full of 'germ ridden
people' (this is not really true of course but it's a fun pun of mine :-) )
Add tmp/TODO.md file with todo items not specific to any open issue.
Release 1.0.46 2023-08-02
Move parse_verbosity(3) to dbg/dbg.c as it is the dbg code that uses the
verbosity_level variable. Ran make test to verify that this works okay.
Updated dbg.3 man page to add this function to the man page. Also improved the
man page in some clarification of other functions (call semantics for example).
New version of dbg facility and the jparse and json parser: respectively
they are "2.11 2023-08-02" and for both jparse and the json parser "1.1.4 2023-08-02".
Remove parse_verbosity(3) from dyn_array(3) facility as it is in the
dbg(3) facility and the dyn_array(3) facility uses the dbg(3) facility.
Update jfmt.1, jval.1 and jnamval.1 for -F option. There might be a
colourful update in the three files.
Typo fix in print_test.c and make a comment that exists in more than one place
consistent in location.
Release 1.0.45 2023-08-01
New version of jfmt, jval and jnamval: "0.0.10 2023-08-01".
Add two new format options to -F option of jfmt, jval and jnamval:
1line which prints output on one line with one space between the tokens and
nows for no extra whitespace. There might or might not be a joke somewhere in
here :-)
Fix formatting in -h usage string of the above mentioned tools as well.
New version of jparse_test.sh: "1.0.3 2023-08-01". The script now runs
print_test instead of the Makefile doing it and thus cluttering up the output.
A thought was to have it run a new script run_print_test.sh but this was not
necessary and would only add more complexity as the script would just have to
run the tool and check for exit status.
Fix jparse.3 man page for the change in the converted and parsed boolean
convenience macros.
Release 1.0.44 2023-07-31
Add -F format option to to jfmt, jval and jnamval. New version for tools
"0.0.9 2023-07-31". color is aliased to colour. Other options are
default and pedantic. Debug output (with DBG_NONE which has to change -
later on) is added based on the optarg. This option has NOT been added to the
man pages. That will come later either later today or another day.
Add missing setting of variables in alloc_jnamval().
Release 1.0.43 2023-07-30
Improve, typo fix and bug fix jparse/test_jparse/print_test.c. Now checks the
file descriptor versions of the printf functions. Added not_reached(); call
after an error function was called. New version "1.0.1 2023-07-30".
Release 1.0.42 2023-07-29
Add jparse/test_jparse/print_test test tool to test if various
print function calls fail. Here "print function call" refers to:
print(3), fprintf(3), dprintf(3), vprintf(3), vfprintf(3), vdprintf(3)
We also test various print-like functions from parse/util.c such as:
fprint(), fprstr(), print(), prstr(), para(), fpara(), fpr(), vfpr()
Added chk_stdio_printf_err(stream, ret) to determine of a
print function call had an error.
All "print function call"s are now followed by a call to
chk_stdio_printf_err() to properly detect if there was a
failure in the "print function call".
Add print_test execution to make test in jparse/test_jparse/Makefile.
New versions of jfmt, jval and jnamval with some minor bug fixes and
enhancements: "0.0.8 2023-07-29", "0.0.8 2023-07-29" and "0.0.8 2023-07-29". The version change synchronises the version number to keep track
of the versions since the date was the same. This does mean that the version for
both jfmt and jnamval jumped from 0.0.6 to 0.0.8 but this seems like a
worthy compromise to keep things together especially as the documented change
for the version update that technically took place on 28 July 2023 was or if it
was is no longer documented.
For jval and jnamval the functions parse_jval_args() and
parse_jnamval_args() have been updated to take the proper struct, a pointer to
argc from main() and a pointer to argv from main(). That is a int *
and a char ***. The functions increment argc/argv past the file name so
that main() can check if there are any args without having to worry about
argv[0]. The function (not main() - the parsing ones) now show debug
information about the args shown. Currently the debug level is 0 so that it's
always printed but this will change later on. As there is no implementation yet
(but see below) there is no list or array or searching of any kind with the
args.
Since the ability to check if any args is now easy and since -i flag with no
args specified means that there are no matches and that indicates exiting 8
jval and jnamval now have this check. That is the extent of any
implementation details until we have discussed output, how jfmt should format
code, search routines are added and numerous other things. This was just an
aside, one might say.
Added XXX comment to the print count function of each tool that points out
that currently the count is not correct. This will be removed in time.
Update jval(1) man page - add exit code 8 to list of exit codes.
Release 1.0.41 2023-07-27
Update json_util_README.md document.
Added missing exit code to jval tool. Updated version string to: "0.0.6
2023-07-27".
Sequenced exit codes.
Release 1.0.41 2023-07-26
New version of jfmt, jval and jnamval: "0.0.5 2023-07-26".
Moved code in jnamval common to both jval and jnamval to json_util.c
like what was done yesterday for jval.
Fixed bug where one could not do:
echo '"test"' | ./jfmt -and the same thing with jval and jnamval. This also fixes the bug where one
could not do:
./jfmt -
"test"
^Dand the same thing with the other too tools. The above two fixes changes the use
of parse_json_stream() to parse_json(). Note that the FILE * is still
needed.
Add to struct json_util the file path of the JSON file as json_file_path.
Check that the path is not an empty file name and exit with error code 3 if it
is in all three tools.
New version of jparse and json parser: "1.1.3 2023-07-26".
Improve how parse_json_stream() deals with NULL filename. This means that the
check for NULL is moved higher up in the function so that if the path is NULL we
set it to - for stdin. This check was in the function already but the purpose
was to make it so that if NULL it is stdin. If it's NULL though it's likely that
the stream is also NULL which means that the check would never get triggered.
The stream can still end up NULL but it shouldn't be a problem if stdin now.
Improve check if file is ready (function fd_is_ready()) in
parse_json_stream(): don't check if stdin as it should be ready anyway and not
doing this seemed to prevent it from reading the file (in some cases).
Release 1.0.40 2023-07-25
New version for jfmt, jval and jnamval: "0.0.4 2023-07-25". Moved common
variables from the structs for each tool (struct jfmt, struct jval and
struct jnamval) to json_util.h and prefix with json_util rather than the
(jfmt|jval|jnamval)_. More work needs to be done here as jval and jnamval
share variables as well but this will be done in another commit.
The structs (jfmt|jval|jnamval)_number are now json_util_number in
json_util.h and the associated code has been moved to json_util.c.
The code for the common -L option to the three tools has also been moved to
the json_util.c file.
Any other options common to the three tools that I have not mentioned here have
also been moved or so it is believed (it is possible some was missed but these
will be located in time if any are missing).
Move variables and functions common to jval and jnamval to json_util
except that only jval currently uses them. jnamval has more options not used
by jval and so it'll take a bit more work to get it done. Some of the code is
slightly different functionally so that code was not moved over. In particular
some options that are common to both tools have it where jval has fewer
option args: -t for example. The macros that have been updated now are as the
new ones in json_util.h.
Release 1.0.39 2023-07-24
Minor fixes to JSON convenience macros that check for a valid or parsed node.
Now they check that item != NULL first. I kept the == true checks that were
added to them for the booleans even though it doesn't match my style simply
because as a macro it might not be clear immediately that it's a boolean so it
seems like it might be a good place to do that.
Fixed warning: incompatible pointer to integer conversion passing 'struct json *' to parameter of type 'enum item_type' in json_util.c.
Add : suffix to parsed/converted boolean in debug output of JTYPE_STRING and
JTYPE_NUMBER (via fprnumber()) in vjson_fprint(). For parsed if
converted is also false add a : else a ,. Print a : after converted flag.
This separates the parsed/converted status from the actual information of the
type. Here's an example log output for numbers:
JSON tree[3]: lvl: 1 type: JTYPE_NUMBER {p,c:-Fddildldi}: value: -179769313486231570814527423731704356798070567525844996598917476803157260780028538760589558632766878171540458953514382464234321326889464182768467546703537516986049910576551282076245490090389328944075868508455133942304583236903222948165808559332123348274797826204144723168738177180919299881250404026184124858368.0
JSON tree[3]: lvl: 1 type: JTYPE_NUMBER {p:FE}: value: 1e100000000
Note for the first line it has p,c: as the number was both parsed and
converted but for the second line it has just p: as the number could not
be converted. The flags follow the :. Now one might argue that the bools are
part of the numbers but the other data is the number itself.
For strings it's likewise just the rest is string data, not number data:
Updated json parser and jparse version strings to "1.1.2 2023-07-24".
New definition of recently removed macro CONVERTED_JSON_NODE:
#define CONVERTED_JSON_NODE(item) ((item) != NULL && (item)->converted == true)Updated jval and jnamval to parse the options -n and -S as a list as
each option can be specified more than once. A list seems the most natural
approach though a dynamic array might be considered later.
New version for both tools: "0.0.3 2023-07-24".
Release 1.0.38 2023-07-23
Fixed jnum_chk by correcting the output of jnum_gen.
Fixed an incorrect prior use of make rebuild_jnum_test
and hand verified the generated jnum_test.c file.
Added large e-notation values to the jnum.testset.
Improved json_alloc() to explicitly initialize critical booleans
and pointers when allocating a new JSON parse tree node.
Both json_process_decimal() and json_process_floating() no
longer set parsed and converted booleans. That task
if performed by the json_conv_number() function.
The json_conv_number() now correctly attempts to convert
a given JSON number in floating point, e-notation and
integer depending on if the result of is_floating_notation(),
or is_e_notation() or is_decimal(), respectively.
Those tests are used to set the is_floating, is_e_notatiion,
and is_integer booleans, also respectively.
Improved JSON debug messages from json_process_decimal() and
json_process_floating().
Simplified how the is_valid boolean arg is used in jparse/jparse.l.
Refer to tree (instead of "node") when referring to the JSON
tree in jparse.l.
Updated convenience macros in jparse/json_parse.h. Now using:
#define PARSED_JSON_NODE(item) ((item)->parsed == true)
#define CONVERTED_PARSED_JSON_NODE(item) (((item)->parsed == true) && ((item)->converted == true))
#define VALID_JSON_NODE(item) (((item)->parsed == true) || ((item)->converted == true))Use of CONVERTED_JSON_NODE(item) changed to CONVERTED_PARSED_JSON_NODE(item).
Fixed "copy and paste" typo in json_tree_print().
Release 1.0.37 2023-07-22
Use of -H in jnamval implies -N.
Release 1.0.36 2023-07-21
Fix bug in vjson_fprint() where numbers that were converted and parsed were
not printed as the check accidentally was using the wrong macro that checks if
converted is true and parsed is false. It should be it checks for both are true
and then the next check in the else if checks if parsed is true and converted is
false.
Move converted bool in struct json_ structs below the parsed bool.
Make jval and jnamval exit with 7 if a number cannot be represented by a C
type. Updated man pages and usage messages. For jnamval this means what used
to be 7 is now 8: no matches found.
For number conversions in jval and jnamval use the macros to check for
converted/parsed booleans.
Release 1.0.35 2023-07-19
Add initial version of man pages of jfmt(1), jval(1) and jnamval(1) to
repo.
Minor bug fixes in jfmt: set default spaces for -L and -Ioptions to 4, not
0, and add missing -q description to usage message. Updated version to "0.0.3
2023-07-19".
Add the three JSON tools jfmt, jval and jnamval to bug_report.sh.
Release 1.0.34 2023-07-18
New jval version "0.0.2 2023-07-18". Option parsing is complete. All that is
left for this is adding the test code for -S and -n (see below for why these
options were not completely parsed).
Fix jval parsing of -S op=str and -n op=num so that the json conversion
routines are used in order to check that everything is okay according to the
JSON spec. What is NOT implemented is the routines to run the comparisons: that
will be a longer function and also has to happen after option parsing is done
for all three tools and then the man pages are written and followed by that
discussion.
Note that if the string starts with a '"' we pass in true for the quote
parameter to the conversion routine and otherwise we do not. It's the
responsibility of the user to ensure that the string is properly formed
otherwise. A possible problem, however, that might need to be addressed, is
whether or not it can compare strings with quotes in it. This can be decided
later.
If jval -c or jval -C is used show a total count of matches. Currently will
be 0 and for -C the output will be incorrect but that's okay as that's all we
have available now and it lets us test the options.
Sequenced exit codes.
Remove any match type from jval. This was added by me as it seemed useful
but since it's added to jnamval I presume that it should not be in jval.
New version of jnamval "0.0.1 2023-07-18". All options should now be parsed
(unless some were missed by accident). Options test code was updated to test the
new bits. Exit codes were changed from the original json_util_README.md (which
has been updated) as 1 is already used for unable to write to output file which
is in all tools now (and since 1 was used for jfmt in that way it seems better
to keep them consistent).
Release 1.0.33 2023-07-17
Updated jval version to "0.0.1 2023-07-17".
Add parsing for all options jval. Test code for -n and -S still needs to
be written but everything should be fine.
Removed unused jval code except that one function which might be useful
later is a placeholder - does nothing functionally.
Removed unused jval structs.
Add check that out file is not stdout before flushing/closing when freeing jfmt
struct.
Release 1.0.32 2023-07-16
Add initial code for jfmt: completely parses options and will dump JSON to
output file if it is not the same as the input file and the file does not
already exist. If not specified it will be stdout.
Add TEMPORARY PLACEHOLDER source and header files for jval and
jnamval tools. The key term that is important is TEMPORARY PLACEHOLDER. The
files are actually copies of what were once jprint related but for each tool
jprint / JPRINT were changed to their new tool counterparts. This is done this
way because some of the code is common and this way they are in the repo. A lot
of code will be removed but some will be kept. Much will also be changed. Option
parsing in particular is useful but this will also have some changes. As it will
take more work to get them in order I have kept the other things in too for now
but this GREATLY simplifies starting the tools which seems like a worthy
thing to do. A very nice way that this simplifies starting the tools is that
the parsing of the JSON file is already done. Along with this some of the
options being the same and some of the structs being the same (though they will
have to be modified and some will be removed) it will help get started.
Respective test scripts for each tool have also been added which are also
updated copies of the old script for jprint. These are not run yet because there
is no point in running them yet and it would only slow down make test.
With the exception of the scripts the required changes will be done soon -
hopefully sometime in the coming week (real weeks start on Monday) - but what
this means is that TEMPORARILY the tools function as the incomplete no
longer useful jprint. By required changes I mean that the unnecessary code will
be removed and option parsing / usage strings will be updated (whether the
option parsing is completed in the next week is indeterminate).
Updated jparse/Makefile and run make depend.
Update .gitignore: fixed references of jprint and add the other tools.
Release 1.0.31 2023-07-15
New JSON parser and jparse version "1.1.0 2023-07-15".
Completely resolve issue #752: 'Enhancement: add parsed boolean to the JSON
parser'. This involved the below changes.
Add more checks to JSON node parsed member so that if converted is false but
parsed is true it's not an error (in the cases where parsed is allowed to be
different it is not an error: if they're not allowed to be different we warn
about it if they are different).
Improve printing of string variables and numbers in vjson_fprint().
Add helper macros CONVERTED_PARSED_JSON_NODE, CONVERTED_JSON_NODE and
PARSED_JSON_NODE and updated jparse.3 man page. The
CONVERTED_PARSED_JSON_NODE checks that both the converted and parsed
booleans are true. The CONVERTED_JSON_NODE checks that the boolean converted
is true and that the parsed boolean is false and the PARSED_JSON_NODE checks
that the parsed boolean is true and that the converted boolean is false.
These macros have been used in in place of checks of foo->converted and
foo->parsed.
Rename function sem_node_valid_converted() to be just sem_node_valid() and
make use of appropriate macro: for numbers it uses the VALID_JSON_NODE and or
others it uses CONVERTED_PARSED_JSON_NODE as both converted and parsed
must be true for non-number nodes.
Fixes in json_util_README.md up through the beginning of jnamval. The rest
of the document and a more thorough check of that which has been checked will
come later.
Release 1.0.30 2023-07-14
New JSON parser and jparse version "1.0.11 2023-07-14".
Note on version 1.0.10: yesterday, 13 July 2023, the versions of both the JSON
parser and jparse were updated but the date was not updated. The version date in
the changes for 13 July 2023 has been updated below to not cause confusion.
Added placeholder functions is_floating_notation(),
is_floating_notation_str(), is_e_notation() and is_e_notation_str() for
issue #752. These functions are NOT called yet except that the _str() versions
call the non-_str() versions. Both return true without any tests at this time
which is why they cannot be called in the parse functions!
Added to each struct json_foo a boolean parsed which means that the value
can be parsed but if false it would indicate it cannot be converted. This is
important because a really big number say a googolplex would fail to be
converted in C but it doesn't mean it's not valid JSON. We don't want to tie the
parser to a big number library so other things will have to be done but we
cannot say that it's not valid JSON.
Add back good json file googolplex.json as it now is considered valid json: if
the error is a range error then we know it's valid JSON so we can set parsed to
true. If it's something else we cannot. THIS IS TEMPORARY! It could be that
there is both a range error and it's an invalid string. There are XXX comments
about this as it needs to change!
Fix detection of '.' with e notation if a '.' found and also whether 'e' was
found in the jnum tests.
Add code to is_floating_notation() and is_e_notation() functions. More needs
to be checked and there are still some XXX comments to look at. Note that the
code here does the same checks as json_process_floating() and this might not
be strictly correct but that can determined later: as it's used only in JSON
parsing it should be okay for the time being.
Check for more than one dot in floating point number strings for JSON and
otherwise. For JSON this is not strictly necessary as the scanner will report an
invalid token but we do it for safety anyway.
Add to function vjson_fprint() code that prints out the string of a number if
converted is false but parsed is true.
Add macro VALID_JSON_NODE that determines if a struct json_foo is valid
which means that either item->converted or item->parsed is true. This macro
is currently used prior to reporting an error in the conversion functions. This
macro is documented in both jparse.1 and jparse.3.
Add function sem_node_valid_parsed() which is like
sem_node_valid_converted() but for the parsed boolean instead of the converted
boolean.
Update jparse.1 man page noting the condition where if a value cannot be
converted but is valid JSON it will still report that it's valid JSON but will
warn that conversion failed.
Release 1.0.29 2023-07-13
New JSON parser and jparse version "1.0.10 2023-07-13".
Fix regexp for JSON_NUMBER in json parser. For exponents it only allowed a
fraction followed by an exponent rather than an integer followed by an exponent.
In other words 1.0e0 was valid but 1e0 was not.
Note that a googolplex is too big for C so the file good/googolplex.json was
removed. For reference a googolplex is
1e10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.
We don't have a directory for the jparse test suite that is although valid JSON
is not possible to parse. We do incidentally have this functionality for
info.json and auth.json files though.
A googol (yes this is where the name 'Google' comes from :-) ) is 1e100 and is
not too big for C.
The file test_ioccc/test_JSON/general.json/bad/info.timestamp-0.json is
actually valid JSON so it was moved to the good subdirectory. Now make test at
the top level directory works fine.
Release 1.0.28 2023-07-03
Fixes and improvements applied to the new
json-README.md document (which was not documented here
yesterday when it was added).
Release 1.0.27 2023-07-02
Bug fixes in jparse_test.sh (erroneously it referred to a variable that did
not exist in the function in question - to do with error location tests - and
also had a log message outside the check of the verbosity level).
Add several new good test JSON files under jparse/test_jparse/test_JSON/good/.
Add another bad JSON test file for both location reporting and otherwise.
In an effort to write the specifications for command line utilities
that read JSON, and to assist others in understanding the JSON
related code, code comments, data structures and documentation found
in this GitHub repo, we add a first draft of jparse/json-README.md.
The modified JSON string encoding rules as given in jparse/json-README.md
have been implemented. The following characters are no longer encoded
and are represented as themselves:
- & is no longer encoded
- / is no longer encoded
- < is no longer encoded
-
is no longer encoded
- [\x80-\xff] is no longer encoded
Release 1.0.26 2023-06-29
New version of jprint "0.0.33 2023-06-29".
Removed 'num' from description of 'simple' for options -t and -Y and instead
say what it translates to (int,float,exp).
Added four new good test json files under test_jparse/test_JSON/good:
42.json, foo.json, just_array.json and top_level_array.json.
Sorted (by category) the member variables in struct jprint as well as in the
function alloc_jprint() (in jprint_util.c).
Added options -o and -r but at this time nothing is done with them: booleans
are set but nothing else. Why is -o before -r? To spell out 'or' which is
what the -o option does! Obviously :-)
Make bug_report.sh check that both jprint and jprint_test.sh are
executable.
Add support of proper matching for JSON booleans and nulls. This does not apply
(just like for the others that are in) for inside arrays - just those by
themselves. Works for -Y to search by value and without -Y to search by
name, showing the name or value respectively, depending on if a match is found.
The -p or -j options can override this of course. It should be remembered
though that nulls and booleans have a string name so only string is checked
before adding a match.
Release 1.0.25 2023-06-28
Add new good JSON test file 42.json under
jparse/test_jparse/test_JSON/good/42.json.
New jprint version "0.0.32 2023-06-28".
Fixed potential memory leak in jprint: although currently none exist the
matches list in the struct jprint (instead of the matches list in each pattern)
was not freed.
Use of jprint -s and jprint -j now possible.
New jparse and json parser version "1.0.9 2023-06-28". Add new function extern char const *json_get_type_str(struct json *node, bool encoded); to jparse
library to get the matched (in the parser/lexer) name that triggered parsing.
Updated man page.
Make use of json_get_type_str() in jprint for modularity. This only applies
to strings and number types.
Release 1.0.24 2023-06-27
New jprint version "0.0.30 2023-06-27".
Fix possible NULL pointer dereference. It should never have happened but it was
theoretically possible. This fix involves a slight change in the way
is_jprint_match() works in that it takes an additional char *: a name that
is the name to match if pattern or pattern->pattern is NULL. If both are
name and either pattern or pattern->pattern are NULL it is an error. This
simplifies checking a bit and there is another use in mind that will have to
come with another commit (if it proves useful; as the concept of name_args is
currently not correct it might be that this won't be useful at all and that the
struct is even removed).
jprint -Y str partially works now. It's not perfect in that in some cases it
can end up matching other types but that depends on the JSON file and options.
The function add_jprint_match() can now have a NULL pattern and also NULL
pattern->pattern. If pattern == NULL then the new jprint->matches is
iterated through and set up. This change is to facilitate adding matches without
patterns for when printing the entire file but this part has not been
implemented yet. More needs to be tested when calling this function but other
things have to be done before that can be done and once again it might be that
the pattern struct and patterns list is completely removed once certain pending
functions are added to the jparse library.
Add back call to run json check tool. Used after reading in the data block but
prior to parsing json. Fix use after free in the function that does this. It
dereferenced the jparse struct after calling free_jparse(). This was not
detected because it happens only when the tool failed and the test case used was
not invalid where I tested it but is in macOS.
Add struct json *json_tree to struct jprint as a convenience. Some functions
are directly passed this but this could be changed if desired.
Fixes in handling of jprint -Q. We have to check not only that it's a string
and printing syntax or quotes are requested but that the type of the match is a
string (JTYPE_STRING) as well. Get rid of string bool in jprint_match
struct as it's not needed with the type.
Improvements in printing some types wrt quotes. Now the function
add_jprint_match() takes two types, a name and value type and the struct jprint_match has a enum item_type name_type and enum item_type value_type.
The determination of which type to use still only has one node at a time so it's
not perfect but it does help somewhat and will be of value later on.
Further update version of jprint to "0.0.31 2023-06-27". The handling of
adding matches of strings and numbers with values and names being different (if
they are in the file) has been implemented. This required a lot of extra code
and more modularity needs to be added but that can come later.
Release 1.0.23 2023-06-26
New jprint version "0.0.29 2023-06-26".
Fix handling of -L and -I. Indenting and printing of levels was messed up.
Doing this I was able to remove a lot of duplicate code too.
Add file test_jparse/test_JSON/good/JSON_misfeature_number_7.json from a
GitHub comment.
Minor update to h2g2.json file - add a boolean to elements of the person
array.
Added stub code for JSON tree search functions.
Added pre-order (NLR) and post-order (LRN) search option to JSON
tree talking. Added post_order arg to json_tree_walk() and
vjson_tree_walk().
Added depth arg to json_tree_walk().
Release 1.0.22 2023-06-25
New jprint version "0.0.28 2023-06-25". Removed -G regex option.
Slight formatting in jprint usage string (to do with aliases).
Rename struct jprint boolean substrings_okay to use_substrings as it seems
a better description and it matches the pattern struct as well.
Modularise vjprint_json_search() by new function is_jprint_match() which
takes the struct jprint *, the pattern, node and string and based on the type
does the right checks. Will not return on NULL pointers. Returns true if it is a
match and false otherwise.
Add more safety when checking for matches by checking that item != NULL and
that item->converted is true and that the string is not NULL.
Clarify that one can use -p both or -p b but not -p n/-p name or -p v/-p value with -j. This was changed yesterday as it's just as easy to
check (and is slightly more user-friendly) that as it is for explicit use of
-p full stop.
Move most functions from jprint.c to jprint_util.c and move structs to
jprint_util.h. This is because most functions are utility in nature. The
jprint_sanity_chks() remains in jprint.c as usage() is the same name as
other tools usage function so it's a static function in jprint.c but
jprint_sanity_chks() uses that function. It could also be renamed to
jprint_usage() of course.
Release 1.0.21 2023-06-24
New jprint version at "0.0.27 2023-06-24". If -j is used don't make use of
-p b or -p both an error. It's only an error if printing of just name or
just value is specified (after the -j as -j will set both). Checking for
this is just as simple as for -p being used at all and it seems slightly more
user-friendly to do it this way.
Modularise option checking of jprint by moving it to the
jprint_sanity_chks() function which now returns a FILE *, the file to read
the JSON from. The function will never return a NULL FILE *. It will not
return with a command line error. It will check all options and verify that the
right number of args have been specified. main() will add the name_args to
the patterns list and go from there if jprint_sanity_chks() returns. As the
argc and argv have to be shifted in main() they are a int * and char ***
respectively rather than their usual int and char **.
Add function parse_jprint_name_args() to iterate through command line, looking
for name_args. This function is called by the jprint_sanity_chks() as some
options have to be checked after looking on the command line for name_args.
Make running JSON check tool more modular which fixes bug of printing output
more than once. The struct jprint has the FILE *json_file, char *file_contents as well as a FILE * for the json check tool stream and char *s for the check tool path and args.
Make jprint.c functions in same order as in jprint.h.
Implement jprint -Q.
Fix jprint -L for some types. Not all types have been tested but numbers,
strings and possibly members are correct. When more is done other types will be
verified too.
Release 1.0.20 2023-06-23
New jprint version at "0.0.26 2023-06-23".
Add booleans (including renaming one uintmax_t to be a boolean and adding a
new uintmax_t in its place) in struct jprint to indicate that specific
options were used rather than relying on the uintmax_ts being not 0.
Add inclusion of string.h and feature test macro _GNU_SOURCE to jprint.h
for strcasestr().
Fix sign comparison warning in util.h.
Fix use after free error in util.c (function vcmdprintf()).
Fix error in jprint_run_tests() causing invalid test result and warning about
unused computed value.
The jprint_match struct (which is still a work in progress) now has two
struct json *s: the node name and node value. The name is what was matched,
whether or not value was searched or not and the value is what will be printed,
either value or name depending on the options. As both might need to be printed
this can also be a misnomer but I believe having the nodes will be useful at
some point.
Improve (but not yet fixed) the value / name added to matches particularly with
substring matching. The value is still the same as the name in the match struct
but with substring matches the match is the pattern (name_arg) specified but
the value (what to be printed) is the actual member found.
Typo fix in debug message. The jprint -o option no longer exists but it was
referenced.
New jparse version "1.0.8 2023-06-23". Fixed display bug in vjson_fprint()
for some of the int types. In particular it cast the booleans to an int rather
than use the as_type members of the struct. This would mean that instead of
printing the actual number it would print a boolean as an integer.
More modularity in jprint printing matches. Several new functions have been
added. These will be useful later for the printing of not just a JSON file if no
pattern requested but also printing matches. More will be added.
Fix typo in usage string with -Y based on the original concept. It takes just
one name_arg but is not an option arg but rather an arg to the program itself.
Only the type is an option arg to the option.
Fix unmatched .RE in jsemtblgen.8 man page.
Add initial man page for jprint. This is currently incomplete. The BUGS
section just says that there are too many things to list at this time as the
tool is in progress and there are a number of things that are incorrect even
besides it. There are some examples but with the exception of one that prints
out the file it is not correct output (and it's not correct for the file in full
depending on options specified). The synopsis is complete and the description is
a good start but the options list is incomplete. There are a lot more to be
added but I want to have the file in and I'll continue working on it in the
coming days.
Release 1.0.19 2023-06-22
New jprint version at "0.0.25 2023-06-22".
Modify behaviour of jprint -c to completely match grep -c (which was my
original idea in the first place). If more than one pattern (as in name_arg)
is specified and found it is a total of the matches found.
Implement jprint -E to such a degree as is currently possible.
Minor improvements to jprint usage string including not hard coding in the
string the default maximum depth to parse, instead using the macro
JSON_DEFAULT_MAX_DEPTH.
Implement jprint -P. When this option is disabled the spaces specified still
separate the tokens if printing both name and value. Default is 1 space, changed
from 0 (it was always supposed to be 1).
When printing both (without json syntax) print on the same line the name and
value.
Added boolean to indicate that -b was used. Currently this is unused but it
will allow not relying on the spaces being not 0 as the default is 1. This fact
is not yet needed but it might be and it's a clear way to indicate if the option
was used or not.
Make matching printing function a bit more modular by introducing a new
function, jprint_print_match(). Note that the concept of patterns is incorrect
and has to be fixed. Discussion is pending.
New jparse and json parser version "1.0.7 2023-06-22". Make print tree function
a bit better in that for null type it prints not NULL but null to indicate
it's a JSON null to remove any ambiguity from C's NULL.
Typo fixes in jprint usage string.
Release 1.0.18 2023-06-21
New jprint version "0.0.24 2023-06-21".
jprint no longer accepts negative max values for options -l and -N. Only
-n allows a max < 0. For -N it makes no sense and for -l it's of
questionable use. That can be decided later but let's just say that if it was a
square root it would be too imaginary :-) One of the jprint -K tests tests
negative max but no others do. In that code only -l was tested but the one for
negative max is now -n. A test case for parsing this error in command line
cannot be done because that's a fatal error which would make the program exit.
It might be possible to do this in the test script at a later time.
Implement -L option for indenting after levels with -j option. The text said
that -L has no effect without -j but I felt like it might be useful to have
it even without -j and the code is already there so for now I have kept it and
removed the text that stated it has no effect without -j. This can be undone
if necessary.
Implement jprint -s (substring match). This means that the only other kind of
matching that has to be added is regexps. It should be noted, however, that
the matching is not entirely complete and some of it will have to change as
well. This holds for exact matches as well.
Implement jprint -I (indent json syntax based on the depth).
Implement jprint -b.
Release 1.0.17 2023-06-20
New jprint version "0.0.23 2023-06-20".
Process write only pipe if -S tool specified. To see details check git log and
search for 'Process write only pipe if -S tool specified'.
Implement -c option to show only count of matches. Use of -o (print entire
file) and -c (show only count) cannot be used together. Use of -c without
specifying any patterns is also a command line error as there cannot be any
matches.
Change debug level 0 messages to debug level 1.
Implement case-insensitive option -i.
New jparse and JSON parser version both at "1.0.6 2023-06-20". Added boolean
to struct json_number a boolean is_integer for integers to simplify checks
in jprint. If integer the booleans for floating points are set to false but if
number string is parsed as floating point only the integer boolean is set to
false. This is because the function to parse floating points should take care of
it (there are a lot of paths in the integer conversions) and that function only
returns true or false without giving an indication if it's an exponent or not.
Don't increment the count of matches if the type is not the same. By type this
refers to the JTYPE_* enum found in json_parse.h.
Add more strict checks for number types based on options.
Disable explicit check for strings in match function as the check there is
problematic anyway and this helps in another way until the function itself can
be fixed. It's very much a work in progress.
Release 1.0.16 2023-06-19
New jprint version "0.0.22 2023-06-19".
Fix level constraints and matching of value versus name constraints. These have
only been tested in very basic JSON files but both now work in principle. There
were a couple problems. See the commit log for details (search for the string,
without quotes, "0.0.22 2023-06-19").
Implement -L option.
Implement -C and -B options if -j specified (-j is required) and partly
implement -j (for simple files). It is not an error if -j is not specified:
it simply has no effect. The struct jprint_match now has a boolean string:
if true it means the match is a string.
Partly process -Y option. It is buggy for strings due to the way the function
is called. There is a workaround to let values be found but it also ends up
letting names be found. This has to be fixed at a later time. The number check
for -Y is incomplete as well and the simple and compound types are also not
checked yet.
Release 1.0.15 2023-06-18
New jprint version at "0.0.21 2023-06-18". jprint now has a matches list per
pattern. The tree traversal functions will search for matches and after that
function finally returns the printing of matches will be found. This will
require additional functions that figure out how to print based on the json type
and jprint options along with the jprint type but for now it prints just the
name and value.
Note that by 'everything is added to the list' mentioned above this means that
name and value will be added as a value. This is something that will be fixed
later but it's the way the tree is traversed that causes this. Be aware also
that for each pattern requested every member of currently supported types will
be added so you can see duplicates. This is because no matching is done yet.
New debug message added. The level is currently 0 so will always be printed but
just like the others this will be changed.
Stop extraneous newlines from being printed.
Add initial support for searching by value or name (value via -Y option). This
will only work for simple json files (or I have only tested simple json files at
this time). If value is being searched we print the name. I'm not sure now if
that's correct behaviour. If it is not then the check will have to be changed.
Checks for basic patterns are added! No regexp support yet. This does not mean
that the printing of name and/or value is yet correct. This needs to be worked
on. Right now a match can happen for either name or value even though it's
supposed to be one or the other.
Rename some functions to match what they do better (they do not print but search
for matches).
Added double blank lines before Release markdown headers in this file.
Rename is_open_stream() to is_open_file_stream() in jparse/util.c.
Added fd_is_ready() and flush_tty() to jparse/util.c.
Added flush_stdin as new 2nd arg to shell_cmd() in jparse/util.c.
Changed pipe_open() in jparse/util.c to only flush stdin if write_mode.
Changed JSON parser to use new fd_is_ready() function.
Changed jprint and jparse/json_sem to use new fd_is_ready() function.
Updated mkiocccentry to not flush stdin when calling shell functions.
Updated txzchk to not flush stdin when calling shell functions.
Release 1.0.14 2023-06-17
New jprint version at "0.0.20 2023-06-17".
Fix special handling for -Y option wrt exactly one name arg. The idea behind the
check is that if -G is specified then one cannot specify a name_arg. If it
is not specified then you must specify one name_arg. If both -G and a
name_arg are not specified it is also an error but there was a bug which
required that -G was used.
Don't show that substrings are ignored for patterns that are regexps (in
jprint).
Sequenced exit codes in jprint.
Add callback function and calling code to jprint. Currently it does not check
for any constraints and will print only json nodes which has a string as the
name or value. The name is not ultimately desired without printing json as a
whole but for now the name is printed. This means that both name and value might
be printed. There are too many newlines printed as well. Much more needs to be
done with these functions.
jprint now checks level (depth) constraints before printing a json member.
This needs to be tested more carefully with bigger documents.
Release 1.0.13 2023-06-16
New jprint version "0.0.19 2023-06-16" - will print entire file if no pattern
specified OR the new option -o is specified but only if valid JSON (for both).
A note that if read_all() fails to read in the entire file then there could be
a problem but we only print the file if the data read is not NULL. This change
is important as it means now all that needs to be done is add the handling of
the JSON checker/semantics tool and then the compiling of regexps (if requested)
and then printing for any matches found (the search routines have to be written
too as does the code to traverse the tree - or 'climb down the tree' :-) ). In
other words one of the features of the program is complete! Note that use of
-o when a pattern is specified is a command line error.
Changed pipe_open() to allow for write mode. Takes a new boolean write_mode
which says to open for write mode rather than read mode.
Fix handling of jprint -S and jprint -A (specifically -S: a segfault in
some conditions namely if the path did not exist which was not checked for and
thus not triggered until this change). Verify path is a regular file and is
executable. Check args and if specified make sure not empty and make sure that
-S is specified as well. Checks -S path first.
If jprint -S path is an executable file then attempt to run, first redirecting
stdout and stderr to /dev/null, checking the exit code. If exit code is not 0
exit as an error. If it is 0 open a write only pipe. The pipe is not used yet
but will be once it is clear how it should be used. At this time the -A args
might be a misnomer or it might be a misunderstanding on my part or it might not
even matter. Currently it's used as options and the options list is terminated
by --. It might be that it's supposed to be args to the program, not options.
Or it might be that it doesn't matter. This is TBD later.
Fix minor bug in txzchk that prevented a debug message from being shown even
if the debug level requested was high enough. New version "1.0.2 2023-11-06".
Release 1.0.12 2023-06-15
Change option letters of jprint a bit as described next.
-s now means search substrings. Conflicts with -g. -s was -M.
-S sets the path to the tool to run, which although in concept should be a
semantics tool checker, it can actually be something else like grep. This option
was -s.
-A sets the args to the -S path. This option was -S. Empty args are not
allowed and use of -A requires use of -S. The difference besides letters is
that args now must not be empty though this might not be strictly necessary.
Allow for both regular expressions and normal patterns via the -G regexp
option. Does not conflict with -s because the regexp is added as a regexp but
the name_args will allow substrings if -s specified - but not for the -G
regexp. -G does not conflict with -Y as long as only -G is used only once
and no additional name_arg is specified at the command line after file.json.
Fix bug where the command line could be messed up and the file.json is thought
to be a different arg specified.
Updated jprint version to "0.0.18 2023-06-15".
Release 1.0.11 2023-06-14
Minor fix in jparse error location reporting. When the erroneous token was at
the very first column the column reported was column 0 which is incorrect by the
standard (de facto or otherwise) and also editors like vim which could be
confusing. New JSON parser version "1.0.5 2023-06-14". Since this also changes
the jparse tool indirectly I also updated the jparse version to be "1.0.5
2023-06-14" (skipping 1.0.4 in order to make it match as this seems like a good
time to make them match version and date).
Renamed test_jparse/test_JSON/bad_loc/comma-eof.json ->
test_jparse/test_JSON/bad_loc/braces-comma-eof.json as it actually has an
otherwise valid json file, {}. Added a comma-eof.json and respective error
file which has the first character being a comma.
Added several other test files (and their corresponding .err file) to have
more tests in JSON error location reporting.
Added another bad json file
test_ioccc/test_JSON/general.json/bad/third-line-backspace.json.
Added make rule in jparse/test_jparse to simplify adding the
jparse/test_jparse/test_JSON/bad_loc error files. To use run from the
jparse/test_jparse directory make rebuild_jparse_err_files. This rule
deletes the error files prior to regenerating them.
jprint version updated to "0.0.17 2023-06-14". jprint -h modified for the
-m max_depth option. jprint -Y now accepts simple type as well.
Rename jprint -S to jprint -M, add new options -s and -S for a tool path
and tool args to be run on file.json. Right now all that is done is the old
processing of -S is now done for -M and -s sets a path and -S sets args.
It makes sure that if -S is specified then so is -s. It does not check yet
if the tool path exists, is a regular file and is executable and obviously it
does not execute anything yet either as this tool is to be written later and the
details are not specific yet either.
Make it so that use of both jprint -j and jprint -M is an error.
Change exit codes for jprint. Specifically 7 is no longer reserved for an
allocation error as it's more of an internal error. 7 is now for the JSON check
tool failure which is related to the options -s and -S. Sequenced exit
codes.
jprint -Y requires exactly two args to the command - the file and one
name_arg. Anything else is an error.
Release 1.0.10 2023-06-13
New jparse version at "1.0.3 2023-06-12" and json parser version at "1.0.4
2023-06-12". The json parser version was bumped by .2 rather than .1 since it
should have been updated the other day when jparse was updated and it might be
that jparse itself did not need to be updated: that's not necessarily true
either way. The significant change here is that when encountering a \t we
increment the column by 8. This appears to be what a tab is but it might not be
perfect as there might be other cases where a \t is some other count. This has
to be tested further.
Update jprint_test.sh and add new files under
jparse/test_jparse/test_JSON/bad_loc. The new functionality, which is only run
if the option -L is passed to the script, tests for the exact match of errors
(in the *.err files) of the *.json files in this subdirectory. These JSON
files have syntax errors where the purpose is to test location reporting. If
there is not an exact match make test will fail. For now there is only one
file but more will be added in time. The purpose of this new feature is to test
the location reporting but we can add more files in time. An important thing to
note is the new README.md file in jparse/test_jparse which explains what one
MUST do after adding a new invalid JSON file under
jparse/test_jparse/test_JSON/bad_loc. This information is also in the man page
jparse(8).
New jprint version "0.0.16 2023-06-13" with new option. Using -Y type one
will be able to search by value instead of name. Currently it does not check for
only one value but this will likely change in a future update. Code does not yet
check that the type matches either. In other words if one specifies -Y int it
does not verify that the name_arg is actually an int. It might not be
necessary either since the JSON parser has the values and names as original
strings. This is TBD later. But an important question has to be answered right
now.
/*
* Why is this option -Y? Why not Y? Because Y, that's why! Why
* besides, all the other good options were already taken. :-)
* Specifically the letter Y has a V in it and V would have been the
* obvious choice but both v and V are taken. This is why you'll
* have to believe us when we tell you that this is a good enough
* reason why! :-)
*/
Release 1.0.9 2023-06-12
New jprint version "0.0.15 2023-06-12".
Make jprint -h exit codes formatting consistent with jparse.
Make struct jprint options in jprint.c a struct jprint *jprint as it will
hold other information besides options including additional structs. Added
free_jprint(struct jprint *jprint) function to completely free everything in
it and then itself.
Fix bug in jprint checking of - for stdin. It shouldn't be just checking the
first char as being - but rather the entire arg. Without this it results in
strange behaviour when say -555 is the file arg.
Added initial jprint_test.sh test script. For now it is very much like
jparse_test.sh except it doesn't test the strings file as jprint doesn't
have an option to read args as a string. Whether it might be good to do this or
to change the script so that it can read from stdin are questions to be answered
at a later date. Something it additionally does is run jprint -K test mode.
Fix incorrect exit codes in help string of jparse_test.sh.
Add patterns / regexps to patterns linked list in jprint struct. For now it
only iterates through the list, showing that a pattern or regexp will be
searched for. Functions to add and free the list added. Patterns are appended to
the list so that the first pattern/regexp specified will be looked for first.
Change exit codes in jprint a bit, making 7 exclusive for memory allocation
errors and >= 15 for other internal errors.
Release 1.0.8 2023-06-11
Fix jparse column location calculations where error messages just showed the
invalid token at column 1 even when the bad token is not on column 1.
Bumped jprint version to "0.0.14 2023-06-11".
When no name_arg is passed to jprint it will print the entire file. It was
never a condition that would cause anything but exit code 0 (assuming the json
file is valid) but it is now more explicit in the code as of 11 June 2023 that
the entire file will be printed once the printing code is implemented.
It is an error to use jprint -p {n,v,b} and jprint -j together.
Move the many options in jprint's main() to struct jprint_options to
organise them and to help with passing to necessary callbacks later on.
Release 1.0.7 2023-06-10
Release jprint version "0.0.13 2023-06-10".
Make jprint -S -g a command line error.
Use of jprint -b [num]t (or -b tab) requires use of -p b (or -p both)
and it is an error if this condition is true.
Add functions to parse -I, -b and -L options.
Use of jprint -j implies -p b -b 1 -e -Q 4 -t any.
But fix check of number arguments after options in jprint. The comparison was
wrong and could cause invalid number of arguments at command line even when they
are correct.
Clarify use of jprint -I and -L options in help string.
Add more test cases of bits.
Fix potential NULL pointer dereference in parsing of number ranges.
Release 1.0.6 2023-06-09
New jprint version "0.0.10 2023-06-09". This version now correctly handles the
number ranges in the form of e.g. -l 5:-3 where if there are 10 matches it
will print the fifth match up through the third to last match.
Fixed seqcexit and picky rules for jprint and sequenced exit codes.
Change semantics of jprint -L option. It now takes an option argument which is
the number of spaces or a tab which will be printed after the JSON level is
printed. Like -b t this has to be fixed in that we cannot assume that t or
tab is 8 spaces instead of \t but this will change later.
Further version update of jprint, now being "0.0.11 2023-06-09". Semantics of
-L and -b changed further. Can be -[b|L] <num>[t|s] where s is optional.
If a number is specified without t it implies spaces. If a s is specified it
implies spaces and if nothing is specified it implies spaces. Anything else is a
syntax error.
Add initial testing of -t type option after fixing an issue with the bitwise
test functions. See jprint_test.c for details.
Release jprint version "0.0.12 2023-06-09": change semantics of -I option to
match that of -b and -L. Finished -t type test code.
Release 1.0.5 2023-06-08
jprint version "0.0.9 2023-06-08". At this time I (Cody) believe all known
checks for jprint options have been added. Also test functions for the number
range code and print mode have been added and run successfully! To run test code
pass in -K to jprint.
The next step is to write some test functions (likely printing debug
messages for different options and their option arguments). Currently (as of
version 0.0.8) the number range test functions are successful.
If all is OK the code to traverse the tree to look for simple matches (this does
not mean the JPRINT_TYPE_SIMPLE) can be added. At first the tool will not
check for the constraints but rather just print the name and value (even though
the default is value (JPRINT_PRINT_VALUE) only I want to make sure that the
ability to to print both is there). This will help make sure that the
traversing works okay before constraints are added. Prior to the following step,
described below, the grep-like functionality, using regex.h, can be added (it
might be better to instead add the grep-like functionality after the below - the
constraints - are added but this will be determined at the time).
Once the above is okay the constraints can be added. The tests should be easier
to do than the traversing and following parts but my hope is that in the coming
days more than tests can be added.
Once this is all done a comprehensive test script can be added to the repo that
is called by the entire test suite (including bug_report.sh) so that we can
verify that jprint works as expected. Then any issues can be fixed and the
tests can be run again. This might take more than a few days but hopefully
things will move along nicely.
Release 1.0.5 2023-06-05
jprint now accepts a -m max_depth option to allow for one to specify maximum
depth to traverse the json tree, assuming it's valid JSON. Defaults to
JSON_DEFAULT_MAX_DEPTH == 256. With debug level DBG_NONE (this will change
to a higher level once the tool is complete or closer to being complete) says
what the limit is, indicating that 0 is no limit and 256 is the default.
jprint now parses the -t type comma-separated option. Currently is a
bitvector but this might change as more is developed (bitvector was not the
first thought).
New jprint version "0.0.6 2023-06-07". It now parses all options and most test
functions for options being set are added as well. This version is backdated to
7 June because this was done on the 7th of June but the version was not updated.
Release 1.0.4 2023-06-04
Split the location table into soup/location_tbl.c and
the location table related utilities into csoup/location_util.c.
Updated location table tbl[] in soup/location_tbl.c
as of 2023 June 05.
Added soup/location utility to look up locations ISO 3166 code(s),
location names, of print the entire location table.
Release initial jprint tool. Current version "0.0.3 2023-06-03". Current
functionality is limited to:
- accepting SOME Of the options but not parsing any except
-h. - if not enough args (at least 1 arg - the file to parse) it prints usage string
(same as-hbut with error message about not enough args). - checks that first arg exists and is a regular file (or
stdin) and if so it
tries to parse it as a JSON file. Depending on status of whether it's valid or
json it will report an error or report valid json. Later it will only report
valid JSON as a debug message. - parsing options except that only the basic option arguments are parsed.
- note that
jprintwill make use of theregex(3)library. - frees the parse tree prior to exiting.
Updated jparse/README.md to note the jprint tool.
MUCH MORE has to be done but this is a great start for a very important tool. We
believe that after the tool is finished and tested (including added to the
overall test suite + the bug_report.sh script!) we (i.e. Cody/@xexyl) can
import the jparse parser and tools into the jparse
repo.
Release 1.0.3 2023-05-22
Added remarks.example.md which has instructions for
writing remarks.md files in the same vein as the
Makefile.example.
Fix bug_report.sh and make clobber (in jparse/) to remove bogus file
jparse/lex.jparse_.c after completion of one of the steps.
Release 1.0.2 2023-04-14
Fix mkiocccentry to write past winner and author handle to the answer file. It
already read from these from the file but did not write them and therefore using
a new answers file did not work at the point that these fields were added to the
tool.
Improved the algorithm for UTF-8 strings to ASCII POSIX safe characters.
The utf8_posix_map hmap[] was changed to no longer map UTF-8 to
ASCII characters '.' NOR '+' NOR '-'. The default_handle() function
was changed to NOT produce more than one '_' (underscore) in a row.
Before these changes, the function produced handles of the form:
^[0-9a-z][0-9a-z._+-]*$Now, handles conform to the following regexp:
^[0-9a-z][0-9a-z_]*$Moreover, default handles will NOT contain 2 or more '_' (underscore)
characters in a row.
Release 1.0.1 2023-02-14
Fix minor shellcheck nits in bug_report.sh and
jparse/test_jparse/jparse_test.sh so that the make prep phase
25 shellcheck will pass as OK.
Test if ctags is available before attempting to run ctags in Makefile rules.
The jparse/Makefile make clobber rule will call the same rule in
jparse/Makefile/test_jparse as well.
Make hostchk.sh grep usage POSIX compliant. This means it no longer uses the
either of the -o or -h options.
Corrected the comment about the Rule 2b value 2503. The previous
comment, as noted by Nicholas Carlini (GitHub user @carlini), was
an out of date comment related to the previous prime 2053 value.
Fix issue in jparse/Makefile where sometimes one would see:
make[2]: `jparse.tab.h' is up to date.
make[2]: `jparse.lex.h' is up to date.
Added jparse_ prefix to json parser so that multiple parsers can exist in
programs that link in the library. New jparse version is 1.0.1 2023-03-10.
Fix jparse/Makefile to have a dependency line for jparse.c (jparse.o). Run make
depend at the root directory.
Change malloc_path() to calloc_path() as it uses calloc(3) not
malloc(3).
Rename macro MAX_NUL_REPORTED -> MAX_NUL_BYTES_REPORTED and allow overriding
it (default is 5; uses #if !defined (MAX_NUL_BYTES_REPORTED).
Rename macro MAX_LOW_REPORTED MAX_LOW_BYTES_REPORTED and allow overriding it
(default is 5; uses #if !defined (MAX_LOW_BYTES_REPORTED).
Fix version of jparse; the prefix added was supposed to be 1.0.1 2023-03-10
and was changed initially but when trying to get a problem resolved it must have
been undone. Now it's instead 1.0.2 2023-03-11.
