Compile 5 suitable datasets to test out the Copypasta test

[andi-halim]

Background

We want to gather 5 datasets in this folder https://drive.google.com/drive/u/0/folders/1OL7toRas_JbCw5nSMbbWb7j6yZciO5r5 that we are confident in in order to limit test and take notes on the copypasta test.

Data Requirements:

• Unique Post Username
• Unique Post Number
• Post Content

Problems

• Find a group of datasets that are diverse enough poses different challenges to the software

Desired Outcome

Be ready to go in testing and filling out a questionnaire regarding each of these datasets.

Tasks

• Collect dataset 1: 24006 Ukraine Russia Tweets
• Collect dataset 2
• Collect dataset 3
• Collect dataset 4
• Collect dataset 5

[DeanEby]

I am not currently able to load in the English and Turkish Misinformation Detection Dataset as it is requiring the message post time which is not in the dataset. I'll generate some filler time data, but I wanted to mention since I thought this was not supposed to be required.

[andi-halim]

@DeanEby responding to your comment from 3 weeks ago! I found the same issue when working through the CLI. I resolved by just selecting some arbitrary column and moving on as I don't believe the test uses that column at all.

Above Doc

@soul-codes Is this a bug that you see as well? We are seeing the copypasta test ask for a message post time column when it's not actually needed.

• this does allude back to what you brought up before about doing the inference + column matching with a dataset before choosing a test and maybe saving those matching settings to a JSON to stick with your project directory. This is probably a more stretch fix but something we could totally prioritize more.