Consider splitting doc from tools into separate repositories

[ahouseholder]

Having the SSVC documents and tool code in the same repository can make it hard to keep tabs on which changes are relevant to which components. It might be time to consider separating SSVC the doc from SSVC tools into separate repos.

[ahouseholder]

Specifically, we have a bit of a tangle in this repository at the moment:

1. The SSVC doc

• the text
• supporting graphics
• CSV files that define the trees
• scripts and tooling to support generating the graphics from the CSV trees

2. SSVC calculator

• JSON schemas
• JSON data conforming to the schema that needs to match the CSVs, but has more data and structure than the CSVs contain
• The calculator UI

3. SSVC python
   • a python module or two that depend on the CSV files

I'm not sure what the right split should be -- the doc and the JSON data/schema seem to be core items, and perhaps the CSVs could be derived from the JSON? The python could also be adapted to depend on the JSON rather than on the CSVs.

However, as seen in #214, there might be some dependency on human editability of the CSVs for customization, which would reverse the JSON -> CSV arrow to become CSV -> JSON. So we'd need to decide on which direction the causal arrow points for that process.

[j---]

The other consideration is that @sei-vsarvepalli has suggested we move to somehow only having one source for the (human readable) definitions, and then generate the PDF and other documentation from that. Presumably a JSON file with all the definitions of the decision points. The benefit of this approach would be that it eliminates manual copy / paste between the markdown and the calculator or other tools where the tools need to make use of the descriptions and definitions of the decision points.
If that is something we wanted to do, would splitting the repos prevent it or make it too hard?

[ahouseholder]

I don't know how that would work. It seems like there is narrative in the paper that doesn't strictly fit into a hierarchy defined by the tree(s).

I wouldn't disagree with the goal of having a set of structured JSON be the authoritative definition from a structural standpoint. As in, if the doc and the JSON disagree, the JSON wins. And that implies that a sizable fraction of the doc could be generated.

But I'm not convinced that the mapping from doc-to-JSON or JSON-to-doc is as clean as it might look, because the doc isn't 100% about defining the structure. It's also about usage, customization, adaptation, application to problems, etc., which doesn't cleanly drop into the way I picture a JSON structure being defined. Plus even if we did map it into a data structure, it seems like it would make the authoring of content considerably worse. The current doc build process is already fragile. While I want that to improve, I'm skeptical of going all-in on the idea that we can just forge a JSON file make everything else be a derivative of that.

[sei-vsarvepalli]

I don't know how that would work. It seems like there is narrative in the paper that doesn't strictly fit into a hierarchy defined by the tree(s).

I wouldn't disagree with the goal of having a set of structured JSON be the authoritative definition from a structural standpoint. As in, if the doc and the JSON disagree, the JSON wins. And that implies that a sizable fraction of the doc could be generated.

But I'm not convinced that the mapping from doc-to-JSON or JSON-to-doc is as clean as it might look, because the doc isn't 100% about defining the structure. It's also about usage, customization, adaptation, application to problems, etc., which doesn't cleanly drop into the way I picture a JSON structure being defined. Plus even if we did map it into a data structure, it seems like it would make the authoring of content considerably worse. The current doc build process is already fragile. While I want that to improve, I'm skeptical of going all-in on the idea that we can just forge a JSON file make everything else be a derivative of that.

Agree with @ahouseholder - not everything will neatly fit into the JSON doc. But descriptions, definitions that are not part of a narrative should in JSON to simplify. We can extend the schema with optional text elements to cover anything missing that is likely to remain in some form integral to each of the already defined Decision Trees. The prose narrative should remain in simple simpler form like Markdown/Text

Hope that clarifies the intent in pursuing JSON to be sort of a System of Record place to hold core information on Decision Trees.

[j---]

@sei-vsarvepalli , is there a JSON-to-markdown script (that we may need to write). I think it's either a pandoc plugin we need to write or a script we need to write, either way.
The md documents are already organized as a set of files that get compiled into one document. It shouldn't be that much harder to structure it so that the definitions of each decision point are their own .md file that is concatenated together at document make time. It might lead to a number of smaller files with the narrative structure, especially in the middle sections. So my file naming hack of using numbers might not survive this choice. But given a JSON to markdown script, I think I can see how the definitions could be inserted into the PDF in the appropriate places from the JSON.