Action plan till 7/28

[tianyin]

@tylergu please fill the action plans like the previous ones and close #135 (btw, if the actions are not done, please always put a note to explain about why).

I create this empty action plan issue to summarize a bit since @tylergu and I were at OSDI and just came back midnight today.

I pinned #132 which provides a detailed guideline of the roadmap of the project.

Operator evaluation

I think, at this point, we should NOT add more new operators. We already have 9 operators. @Essoz is working on the KNative operator. @taham0 and @featherchen are working on two. So, if everything goes well, we will have 12 operators in our evaluation, which is more than what Sieve did.

The KNative Operator is very challenging (as it is different from other operators we evaluated). OTOH, it is interesting as it shows us different code. Thanks @Essoz for working on it.

@taham0 and @featherchen : Thanks for working on evaluating the two operators. We want every student to start from a solid end-to-end evaluation that applies Acto on a chosen operator and goes through the process of finding bugs and reporting them. It could help you to understand the value, and more importantly, the limitations of Acto. Some of them are being addressed by some efforts that have not been merged to the master branch. After you are done with the evaluation, you will have your own ideas and judgements on how to improve things.

@Essoz @unw9527 Please actively help @taham0 and @featherchen if they have any questions -- now you are the senior people in the project :)))

Other important tasks

@tylergu I know the dependency analysis is challenging and you are working hard to address it. OTOH, there are other equally, if not more, important tasks in #132 especially the Domain-Specific Support and Multi-input Testing. Those, from the paper perspective, could be even more important. We should start to plan on solving them.

Summary

We should aim for (1) finding more bugs and (2) finding them faster in the 10 (or 12) operators, instead of increasing the numbers by adding new operators.

[tylergu]

@tylergu

• Categorize field dependency false alarms that can be addressed by direct dependency
• Multi-input testing
  • Modify the input model to generate multiple tests at the same time. The technical challenge here is to ensure the test cases do not conflict with each other (e.g. insert an item to the array field will conflict with deleting that array field.)
  • If the input is rejected, need to run binary search so that no all test cases are rejected
• Postmortem error message check
  • Do not stop testing when seeing error messages
  • Count the error messages as a postmortem step and use their counts as the confidence level of whether this is benign or not.
    • Challenge of this is to recognize which error messages are the same error message: although these error messages are essentially the same error message, the content may be slightly different containing some runtime information. One idea is to group them by line number, however, different error messages maybe thrown at the same line.

---

@Essoz

• Port knative-operator
  • Fix namespace dependency issue
  • Figure out how to deal with conversion webhooks -- errors returned from conversion webhooks lead acto to crash
  • Figure out a way to specify images using tags instead of digest - preload_image() does not work with images pulled using digest due to a bug in containerd
    - [ ] The above item is a limitation of Acto, document it.
  • Inspect results (time permitting)
• Discuss with @tylergu to handle program analysis corner cases with xtradb
• Write bug report for all bugs in xtradb and provide fixes if possible
• Proofread and send all bug reports (redis and xtradb)

---

@unw9527

• Report a bug in CString in Go
• Write unittest for the PR
• Wrtie a PR for storage under pump and tiflash in TiDB
• Test whether invalid input regex is still needed
• Write bug report for all bugs in TiDB and provide fixes if possible
• Proofread and send all bug reports (TiDB and cockroachDB)
• Reply to [BUG] Updating the field spec.ingress.sql.tls.secretName is not reflected in the sql ingress object cockroachdb/cockroach-operator#920

[tianyin]

This action plan is incredibly detailed and informative. Thank you for doing it!!!