v8.3.1: Fix crash with empty RemesPath queries; properly raise error for more bad RemesPath queries

[8.3.1] - 2024-12-22

Fixed

1. Fixed bug where running a RemesPath query in the tree view would cause a crash if the query box was empty or contained only whitespace.
2. Fixed bug where some errors would not be raised on some syntactically invalid RemesPath queries containing two valid items not separated by a binary operator. For example, 1 * -2 7 and bar + @[0] 9 both had this problem.

v8.3: *fully fix* decimal formatting, allow parse \u0000 and \x00, minor improvements

[8.3.0] - 2024-12-22

Changed

1. Use a different algorithm for representing decimal numbers to fix issues 83 and 81 on computers where v8.2 had not fixed those issues.
   • As an unfortunate consequence of this fix, a very very small percentage of very high-precision decimal numbers will lose precision using this new algorithm. The vast majority of numbers with 17 digits of precision (the maximum precision attainable by JsonTools) will not lose precision when reformatted by JsonTools. For example, JsonTools will pretty-print -6553693.3617752995 as the lower-precision -6553693.3617753. However, note that JsonTools still handles high-precision doubles better than the JSON-Viewer plugin.
2. Automatic parsing after editing and automatic JSON schema validation on opening a file no longer makes the error form visible if the user had previously opened the error form and then hid it.
3. Tree view icon and menu item check are now correctly toggled depending on whether a tree view (other than the grepper form's tree view) is visible. This doesn't matter much, since multiple tree views can be open and a tree view that is not visible could become visible again, but it is a nice visual cue.

Fixed

1. When a file is pretty-printed or compressed or edited with RemesPath, the caret will be scrolled into view. Previously, if the file contained very long lines and word wrapping was turned off, the user might have to manually scroll to the left after pretty-printing or compressing.
2. NUL characters (when rendered as \u0000 or \x00) can now be parsed. Previously they raised a fatal error.

v8.2: fix decimal formatting, rename tree views, many bugfixes

[8.2.0] - 2024-11-09

Added

1. When a file is renamed, the name of a tree view associated with that file also changes to match the new name. This also happens when a treeview is refreshed with JSON from a different file.
2. Add rand_schema RemesPath function to generate random JSON from schema.

Changed

1. If a number string is too large or small for a 64-bit floating point number (for example, -2e700, 3.5e+450), the JSON parser will now represent them as -Infinity (if they have a leading - sign) or Infinity, rather than representing them as NaN. The linter also has a new message for when this happens.
2. If a JSON lines document is not compliant with the strict JSON specification, the status bar will now reflect that it is JSON lines.
3. Stop considering the user's Windows culture when determining the UI language for JsonTools (fix issue 82). Now JsonTools will be translated into language X if and only if Notepad++ is in language X.

Fixed

1. Fix issue (introduced in v8.1, see issues 83 and 81) where decimal numbers were given unnecessarily precise string representations. For example, in JsonTools v8.1, 11.11 would be represented as 11.109999999999999 even though the original 11.11 was an equally valid representation. Now all decimal numbers will be given the more compact representation used in v8.0 and earlier unless the more verbose representation introduced in v8.1 would be necessary to avoid loss of precision (so there will be no regression on issue 78). Note that this new algorithm for formatting decimal numbers is about twice as slow as the algorithm used in v8.1 (for 64-bit JsonTools; 32-bit has no performance loss), although the impact on performance will be much less dramatic unless you are compressing an array where almost every element is a non-integer decimal number.
2. Fix the following issues with random string from regex:
   • It previously incorrectly flagged some valid regular expressions (e.g. (?-i)(?:xy{1,2}){,2}) as having two consecutive quantifiers.
   • It previously did not correctly handle some character sets where the final character was - (for example, [+-] previously would only generate +, and now it correctly has a 50% chance of generating - or +)
3. Fix issue where RemesPath incorrectly inferred the type of (a function fun followed by indexers) to be the return type of fun. For example, running the query sum(dict(items(@)).a) on the JSON {"a": [1]} now correctly returns 1.0, but RemesPath used to raise an error because it assumed that dict(items(@)).a had the same type as dict(items(@))
4. Fix very rare crash bug when using the Value to clipboard option of the tree node right-click context menu.
5. Fix bug where some invalid JSON Lines documents (for example, [1, \n2][3]) would be accepted by the JSON Lines parser despite having elements that span multiple lines.
6. Fix minor bugs with how headers are formatted in the s_csv RemesPath function and JSON-to-CSV form.
7. Fix bug where renaming a file subject to schema validation based on filename patterns would cause its tree view to be lost.
8. Fix bug where plugin actions (mainly RemesPath queries in regex mode) that set the text of the entire document to an empty string would not do anything. Those actions will now correctly remove all the text in the document.
9. Fix issue 79 where, on higher display resolutions than the default 125% normally used by molsonkiko in development, some of the advanced controls in the find/replace form would not be visible.
10. Fix annoying but harmless bug where, if the user had two views open and ran a plugin command on one or more selections in the second view, the indicator that JsonTools uses to remember selections (which is supposed to be hidden at all times) would cause the selections to be underlined.
11. Address issue 80; now error form will be cleared of errors when the JSON parser does not log any syntax errors.
12. Make it so that JSON Lines documents are correctly formatted as JSON Lines after sorting by the sort form.
13. Make it so that refreshing a tree view originally associated with oldFile with JSON from newFile will correctly cause the tree view to be associated with newFile. Previously, the tree view stayed associated with oldFile, so attempting to open a new tree view for oldFile would instead close the tree view now associated with newFile, because the command to open a tree view for a file closes that file's tree view if it already has an open one.

v8.1: improve "JSON from Files and APIs" form; improve translations; random string from regex; fix many bugs

[8.1.0] - 2024-08-23

Added

1. Progress reporting with the JSON from files and APIs form (henceforth the grepper form).
2. In the grepper form, pressing Enter inside the Previously viewed directories... box causes the current text of the box to be searched, assuming that it is a valid directory.
3. Translation of the following:
   • Settings in the Settings form.
   • JSON syntax errors and JSON schema validation errors (under the jsonLint field of the translation file)
   • Generic Windows message boxes (the boxes with Ok, Yes, No, and Cancel buttons and no other controls).
4. The path_separator setting for formatting keys/indices and paths. Addresses issue 69.
5. Make it so left-clicking on the Key/index to clipboard and Path to clipboard options of the treenode right-click context menu gets the path or key/index in whatever the default is from your settings, without having to click on one of the sub-menu items.
6. The grepper form now supports \ and / (path separators) in search patterns, as well as ** to match any number of characters (including \).
7. Generation of random strings from regular expressions.
8. The grepper form now reads and parses all files asynchronously, and can be canceled.
9. While the grepper form is working on a request, it now ignores clicks on the API request button and the Search directories button.

Changed

1. Rename Choose schemas to automatically validate filename patterns to Validate files with JSON schema if name matches pattern, in the hopes that the new name will be less confusing.
2. Changed the wording of many JSON syntax error messages to be more consistent, per conky77's suggestion here.
3. When attempting to translate to other languages, JsonTools now checks the UI language of Notepad++ before checking the Windows UI culture.
4. When automatic validation after editing is turned on, only modifications that change the text of the document will trigger re-parsing.

Fixed

1. If there would be an OutOfMemoryException due to running out of memory while formatting JSON (a likely occurrence when using the grepper form), that error is caught and reported with a message box, rather than potentially causing Notepad++ to crash.
2. Ensure that hitting the down key does nothing when the last row of the error form is selected.
3. Fix bug with random json from schema when parsing a schema that has the enum key but not the type key.
4. Make it so JsonTools simply does nothing rather than causing Notepad++ to crash when attempting to run plugin commands on files with more than 2147483647 bytes.
5. Fixed number precision bug (issue 78).
6. Rare bug generating schemas from JSON, only seen with some arrays of objects.
7. Rare crash when saving schemasToFnamePatterns.json if automatic validation after editing is enabled.
8. Changes to the max_schema_validation_problems setting will apply immediately, including to previously compiled JSON schemas.
9. Unrecoverable crash due to stack overflow when generating random JSON from a recursive schema.

v8.0: add translation to other languages; improve tree view UI; remove some minor settings

[8.0.0] - 2024-06-29

Added

1. Made it possible to translate JsonTools into other languages.
2. Made the tree view font size configurable with the tree_view_font_size setting. Fixes issue 66.
3. Added dark mode tree view icons (addresses this GH issue comment).
4. Allow users to manually enter directory names in the JSON from files and APIs form (referred to as "grepper form" for the rest of this section) rather than using a dialog or the dropdown menu.
5. Added a "Search directories" button to the grepper form.

Changed

1. Removed the max_threads_parsing setting for the grepper form, because the underlying implementation was changed in a way that made that setting useless.
2. Removed the allow_datetimes setting, because I do not feel confident that the potential benefits of the setting outweigh the danger of accidentally changing the format of people's dates or datetimes in some way that they were not expecting.
3. When all files have been parsed in the directory chosen with the grepper form, the directory name is no longer reset to the default value.
4. Added a hard limit to the total combined size of all files parsed by the grepper form, to avoid hitting memory errors.

Fixed

1. Avoid unnecessarily refreshing the styles of all forms when settings other than use_npp_styling or tree_view_font_size are changed.
2. The list of recently chosen directories in the grepper form is pre-filtered for existing directories, to hide the partial directory names that contaminate the config file.
3. Fix bug where, if a setting in the config file had an invalid value (for example, a numeric setting having a value of blah), there might be an uncaught exception that would cause Notepad++ to crash. This bug appeared to be most likely to occur when the localization is not set to en-us.

v7.2: fix various bugs; add "Check JSON syntax now" command

[7.2.0] - 2024-04-19

Added

1. Check JSON syntax now command.

Changed

1. Made it so that reloading the error form by pressing Enter would not cause certain message boxes to appear, to eliminate a potential "infinite" loop where the user would hit Enter to close the message box, and that moved focus back to the error form, which then repeated the cycle when they lifted the Enter key.
2. Automatic validation when auto_validate is true no longer opens the prompt asking if user wants to open the error form (if offer_to_show_lint is true), because that could cause Notepad++ to crash or hang forever (see issue 60).
3. Made it so that automatic JSON schema validation (that is, any validation not manually invoked by the plugin menu command) no longer causes the caret to move to the location of the first schema validation error.
4. Automatic validation (including non-schema validation) now refreshes the error form.

Fixed

1. Minor bug in PPrint remembering comments algorithm implementation that caused some arrays and objects to be compressed when they should have been pretty-printed.
2. Fix bug where tests could crash under some circumstances due to filesystem weirdness making it impossible to find test files.

v7.1: improve selection-based mode, fix issues with Notepad++ 8.6.5+, fix many other bugs

Added

1. minLength and maxLength keywords are now considered when making random JSON from schema
2. JsonTools now automatically navigates to the location of the fatal error when it fails to parse a document, unless the attempted parse was auto-triggered (say, by the automatic parse after editing)
3. Selection-based mode now supports any number of remembered selections, and undo and redo actions usually do not cause selections to be forgotten. Performance also appears to be better.

Changed

1. If the location of an error is between the CR and the LF of a CR LF newline, JsonTools will now always move before the CR, rather than placing the caret in between them, which could cause confusion.
2. Automatic JSON schema validation after editing does not happen when the document is in REGEX mode.
3. Removed the max_tracked_json_selections setting, as it is no longer necessary.

Fixed

1. Using the Notepad++ find/replace form in Notepad++ versions 8.6.3 and 8.6.4 now appropriately shifts remembered selections when in selection-based mode.
2. Bug where the space key did not work on the regex search form.
3. Bug where automatic JSON schema validation after editing sometimes caused a deadlock that disabled some functionalities until Notepad++ closed.
4. Possible plugin crash due to index-out-of-bounds error during parsing of JSON Lines.

v7: json schema validation finds multiple errors, better pretty-print w/ comments, regex form guesses CSV parameters, RemesPath improvements

[7.0.0] - 2024-02-09

Added

1. PPrint-style pretty-printing that remembers comments
2. Add and and or non-vectorized functions in RemesPath, which both use conditional excution.
3. Add s_format vectorized RemesPath function, for easy reformatting of JSON strings.
4. Python-style single-line comments in RemesPath
5. A RemesPath user-defined language (UDL) file, providing some very basic syntax highlighting. It is buggy, but that is because the UDL system is inherently buggy, not because I did anything wrong (as far as I know).
6. A : character between two key-value pairs in an object no longer causes a fatal error that makes the parser quit.
7. Add new auto_try_guess_csv_delim_newline setting. If this is true (default false), Regex search form now makes a very basic attempt to "sniff" if the current file is CSV whenever it is opened, or when the Parse as CSV? button is toggled on.

Changed

1. Support for multiple JSON schema validation problems
2. All RemesPath regular expressions are now multiline, meaning that ^ and $ now match the start and end of lines respectively, rather than the start and end of the document.
3. The ifelse vectorized function in RemesPath now uses conditional execution.
4. Default value for sort_keys setting is now false, meaning keys are left in their original order by default. This will not change existing settings.
5. Add optional arguments to stringify non-vectorized function in RemesPath, so that users can control the format of the output.
6. Make dark mode icons darker.
7. This change only affects the code base, not the public API: changed almost all snake_case variable names to camelCase. RemesPath functions still use snake_case (e.g., s_mul and group_by still have those names), and all the settings in Settings.cs (e.g., use_npp_styling) that were previously snake_case are still snake_case.
8. Automatic linting after edits is now disabled while in selection-based mode.
9. Automatic linting after edits will always attempt to parse the entire document, even if the user has made a selection that could be parsed as JSON.
10. Numbers with unnecessary leading 0's (like 01 or 002.5) are now logged at the BAD level, and numbers with trailing decimal points are now logged at the JSON5 level.
11. Error form keypress triggers now execute when the key is released, rather than when it is depressed.
12. Automatic JSON schema validation now ignores the user's selections and always validates the entire document.

Fixed

1. Fixed issue where vectorized functions in RemesPath were not vectorized across objects if the first argument was a function of input and at least one of the non-first arguments was also a function of input.
2. Fixed issue where entering invalid text into comboboxes in the regex search form and grepper form could cause a (mostly harmless) plugin crash.
3. Improved refresh and loading speed for tree viewer on very large files in regex mode.
4. Fix issue where tree view tracking of capture groups in regex search mode was wrong in some cases involving non-ASCII characters.
5. Hitting Escape in the error form moves focus to the editor component.
6. When logging errors in selection-based mode, the error form now navigates to the location of the error in the overall document. Previously it navigated to the location of the error relative to the start of the selection.
7. The regex search form no longer closes the currently active tree viewer when you focus or open it.
8. Register forms (other than the modal AboutForm and JSON-to-CSV form) with Notepad++ using NPPM_MODELESSDIALOG. This permanently fixes the regression in Notepad++ 8.6.1 where Ctrl+X and Ctrl+C stopped working in textboxes.

v6.1.1: improve error form, f-strings and new funcs in RemesPath, better RemesPath error msgs, fix misc bugs

[6.1.1] - 2023-12-28

Fixed

1. Eliminated potentially unrecoverable plugin crash when JSON parser tries to parse document with - or + not followed by numeric chars (e.g. -a, +). Now such badly formatted numbers are parsed as NaN.

[6.1.0] - 2023-12-28

Added

1. Python-style f-strings in RemesPath.
2. s_cat RemesPath non-vectorized function.
3. s_lines, s_lpad, s_rpad, and zfill RemesPath vectorized functions.
4. Select all children treenode action now works for root treenode in JSON Lines and selection-based documents.

Changed

1. RemesPath syntax errors (anything caught by the lexer) now use >>>HERE>>> before the character where the error occurred, similar to how the Notepad++ find/replace form indicates the location of a regular expression syntax error.
2. Not part of public-facing API: Renamed the JQueryContext.Evaluate method to JQueryContext.Operate, and renamed JMutator.Mutate to JMutator.Operate.
3. Make it so automatic validation does not require the document to be re-parsed as JSON, and is suppressed when in regex or ini mode.

Fixed

1. Eliminated plugin crash when attempting to open the regex search form after it had been closed.
2. Greatly improved error form reloading performance.
3. Some UI test failures (and probably related weirdness in public API) on older NPP versions
4. Fix bug where s_csv RemesPath function did not properly handle delimiters that were regex metacharacters like |

v6: TREE VIEW WORKS FOR REGEX SEARCH AND CSV, parser improvements, error form improvement, many bugfixes

Added

1. Option to customize which toolbar icons are displayed, and their order.
2. New regex search form for using treeview to see regex search results in any file.
3. New document type list box in tree view
4. For loops in RemesPath
5. bool, num, s_csv and s_fa RemesPath vectorized arg functions
6. randint, csv_regex, set, and to_csv RemesPath non-vectorized arg functions
7. Make second argument of s_split RemesPath function optional; 1-argument variant splits on whitespace.
8. Right-click dropdown menu in error form, allowing export of errors to JSON or refreshing the form.
9. The JSON parser is now much better at recovering when an object is missing its closing '}' or an array is missing its closing ']'.
10. Support for JSON Schema validation of enum keyword where the type is missing or an array.
11. Ctrl+Up now snaps to parent of currently selected node in tree view. Ctrl+Down now snaps to the last direct child of the currently selected node.

Changed

1. The internal representation of object keys has changed to allow unescaped strings as keys (THIS DOES NOT AFFECT THE PUBLIC API EXCEPT IN SOME CORNER CASES IN REMESPATH).
   • What I mean by this is that previously the key in the JSON object {"\"": 3} was previously internally represented as "\\\"", with the quote character escaped as shown.
   • This had the advantage of making it slightly faster to display object keys (e.g., when pretty-printing/compressing/dumping), but made it impossible for certain strings to be valid object keys, in a way that could not be detected except when pretty-printing or compressing.
   • Under the new system as of this version, all strings are acceptable as object keys. This has positive implications for RemesPath, as it means that users do not need to remember to escape string JNodes before using them as keys in an object.
2. When using the JSON-to-CSV form to create CSV files, newline characters will no longer be escaped in strings. This is one of several changes made in order to acheive compliance with RFC 4180.
3. Made offer_to_show_lint setting (which controls whether a prompt is shown when errors are found) true by default, so that a fresh installation will show the prompt.
4. Change RemesPath indexers to reduce the number of backslash escapes needed to get keys containing special characters like "a\\b" or "\"foo\"\tbar". For instance, previously @.`\\n\\\\a\"` would be required to match the key "\n\\a\"", whereas now @.`\n\\a"` matches it.
5. Running a RemesPath query only causes an attempted re-parsing of the document if the treeview's current file is open.
6. Running a replace query on the find/replace form now causes the tree to display only the values that were mutated.
7. Changed the tabstop order of some forms, made it so that the find/replace form has complete tabstop coverage, and eliminated some tabstop-related issues associated with combo boxes in some forms.
8. Benchmarks for compiling RemesPath queries now more accurately represent the effect of caching on performance.

Fixed

1. Fixed plugin crash when attempting to parse too-large hex numbers like 0x100000000000000000000. Now the parser will fatally fail and add a lint indicating the issue, but the plugin will not actually crash.
2. Fixed some weird issues where mutating a variable in RemesPath could cause re-executing a query on the same input to return a different value. A minimal example: var x = 1; x = @ + 1; x would return 1 + (the number of times the query was executed) prior to this fix, but now it will always return 2 as expected. This was also true of a bunch of other things in RemesPath, including projections and the map operator.
3. Fix issues where running a RemesPath query with a projection that referenced a variable indexing on a compile-time constant would cause an error. For example, var x = @; 1->x should return @ (the input to the query), but prior to this fix, it would instead cause an error.
4. Running tests would previously cause clipboard data to be lost irreversably. Now, if the user's clipboard contained text before running tests, the contents of the clipboard are restored to their pre-test values rather than being hijacked. Non-text data that was copied to the clipboard is still lost when running tests, and I may try to fix that in the future.
5. dict function in RemesPath previously had a bug that could create invalid JSON if the strings to be turned into keys contained special characters (e.g., literal quote chars, \r, \n).
6. access violations when loading error form
7. unnecessary prompt when manually reloading error form
8. issue with trying to view error form when the error form was already open
9. RemesPath backtick strings now can have a literal \ character just before the closing backtick. Previously this was impossible because of a regex-writing bug.
10. Eliminated plugin crash when attempting to validate with an invalid JSON schema. Now a message box will show in that situation.