Skip to content

Commit

Permalink
ENH: Add option to extend default info
Browse files Browse the repository at this point in the history
  • Loading branch information
kamil-certat committed Jul 9, 2024
1 parent 07e44d6 commit af2b8ce
Show file tree
Hide file tree
Showing 4 changed files with 53 additions and 7 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
- Allow saving messages in bulks instead of refreshing the feed immediately (PR#2509 by Kamil Mankowski).
- Add `attribute_mapping` parameter to allow selecting a subset of event attributes as well as additional attribute parameters (PR#2509 by Kamil Mankowski).
- Add `event_separator` parameter to allow keeping IntelMQ events in separated MISP Events based on a given field (PR#2509 by Kamil Mankowski).
- Add `additional_info` parameter to extend the default description of MISP Events (PR#2509 by Kamil Mankowski).
- `intelmq.bots.outputs.smtp_batch.output`: Documentation on multiple recipients added (PR#2501 by Edvard Rejthar).

### Documentation
Expand Down
27 changes: 24 additions & 3 deletions docs/user/bots.md
Original file line number Diff line number Diff line change
Expand Up @@ -4629,9 +4629,30 @@ as not usable for IDS.

**`event_separator`

(optional, string): If set to a field name from IntelMQ event, the bot will group incoming messages
in separated MISP events, based on the value of this field. The `interval_event` parameter acts
for all grouping events together.
(optional, string): If set to a field name from IntelMQ event, the bot will work in parallel on a few
events instead of saving all incomming messages to a one. Each unique value from the field will
use its own MISP Event. This is useful if your feed provides data about multiple entities you would
like to group, for example IPs of C2 servers from different botnets. For a given value, the bot will
use the same MISP Event as long as it's allowed by the `interval_event`.

**`additional_info`

(optional, string): If set, the generated MISP Event will use it in the `info` field of the event,
in addition to the standard IntelMQ description with the time frame (you cannot remove it as the bot
depends of datetimes saved there). If you use `event_separator`, you may want to use `{separator}`
placeholder which will be then replaced with the value of the separator.

For example, the following configuration can be used to create MISP Feed with IPs of C2 servers
of different botnets, having each botnet in a separated MISP Events with an appropiate description.
Each MISP Event will contain objects with the `source.ip` field only, and the events' info will look
like *C2 Servers for botnet-1. IntelMQ event 2024-07-09T14:51:10.825123 - 2024-07-10T14:51:10.825123*

```yaml
event_separator: malware.name
additional_info: C2 Servers for {separator}.
attribute_mapping:
source.ip:
```

**Usage in MISP**

Expand Down
11 changes: 7 additions & 4 deletions intelmq/bots/outputs/misp/output_feed.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,11 @@ class MISPFeedOutputBot(OutputBot, CacheMixin):
bulk_save_count: int = None
misp_org_name = None
misp_org_uuid = None
output_dir: str = (
"/opt/intelmq/var/lib/bots/mispfeed-output" # TODO: should be path
)
output_dir: str = "/opt/intelmq/var/lib/bots/mispfeed-output" # TODO: should be path
_is_multithreadable: bool = False
attribute_mapping: dict = None
event_separator: str = None
additional_info: str = None

@staticmethod
def check_output_dir(dirname):
Expand Down Expand Up @@ -141,10 +140,14 @@ def process(self):

def _generate_new_event(self, key):
self.current_events[key] = MISPEvent()
self.current_events[key].info = "IntelMQ event {begin} - {end}" "".format(
info = "IntelMQ event {begin} - {end}" "".format(
begin=self.min_time_current.isoformat(),
end=self.max_time_current.isoformat(),
)
if self.additional_info:
info = f"{self.additional_info.format(separator=key)} {info}"

self.current_events[key].info = info
self.current_events[key].set_date(datetime.date.today())
self.current_events[key].Orgc = self.misp_org
self.current_events[key].uuid = str(uuid4())
Expand Down
21 changes: 21 additions & 0 deletions intelmq/tests/bots/outputs/misp/test_output_feed.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,27 @@ def test_event(self):
objects = json.load(f).get("Event", {}).get("Object", [])
assert len(objects) == 1

def test_additional_info(self):
self.run_bot(parameters={"additional_info": "This is my custom info."})

current_event = open(f"{self.directory.name}/.current").read()
with open(current_event) as f:
info: str = json.load(f).get("Event", {}).get("info", "")
assert info.startswith("This is my custom info. IntelMQ event ")

def test_additional_info_with_separator(self):
self.run_bot(
parameters={
"additional_info": "Event related to {separator}.",
"event_separator": "malware.name",
}
)

current_events = json.loads(open(f"{self.directory.name}/.current").read())
with open(current_events["salityp2p"]) as f:
info: str = json.load(f).get("Event", {}).get("info", "")
assert info.startswith("Event related to salityp2p. IntelMQ event ")

def test_accumulating_events(self):
self.input_message = [EXAMPLE_EVENT, EXAMPLE_EVENT]
self.run_bot(iterations=2, parameters={"bulk_save_count": 3})
Expand Down

0 comments on commit af2b8ce

Please sign in to comment.