intelmq.bots.parsers.microsoft package

Submodules

intelmq.bots.parsers.microsoft.parser_bingmurls module

Parses BingMURLs data in JSON format.

intelmq.bots.parsers.microsoft.parser_bingmurls.BOT

alias of MicrosoftBingMurlsParserBot

class intelmq.bots.parsers.microsoft.parser_bingmurls.MicrosoftBingMurlsParserBot(*args, **kwargs)

Bases: ParserBot

Parse JSON data from Microsoft’s Bing Malicious URLs list

parse(report: Report)

A basic JSON parser. Assumes a list of objects as input to be yield.

parse_line(line, report)

A generator which can yield one or more messages contained in line.

Report has the full message, thus you can access some metadata. Override for your use.

recover_line(line: dict)

Reverse of “parse” for single lines.

Recovers a fully functional report with only the problematic line by concatenating all strings in “self.tempdata” with “line” with LF newlines. Works fine for most text files.

Parameters:

line (Optional[str], optional) – The currently process line which should be transferred into it’s original appearance. As fallback, “self._current_line” is used if available (depending on self.parse). The default is None.

Raises:

ValueError – If neither the parameter “line” nor the member “self._current_line” is available.

Returns:

str

The reconstructed raw data.

intelmq.bots.parsers.microsoft.parser_ctip module

Parses CTIP data in JSON format.

Key indicatorexpirationdatetime is ignored, meaning is unknown.

There are two different variants of data

• Interflow format: JSON format, MAPPING

• Azure format: JSON stream format, a short example structure:

  {
    "DataFeed": "CTIP-Infected",
    "SourcedFrom": "SinkHoleMessage|SensorMessage"",
    "DateTimeReceivedUtc": nt time
    "DateTimeReceivedUtcTxt": human readable
    "Malware":
    "ThreatCode": "B67-SS-TINBA",
    "ThreatConfidence": "High|Medium|Low|Informational", -> 100/50/20/10
    "TotalEncounters": 3,
    "TLP": "Amber",
    "SourceIp":
    "SourcePort":
    "DestinationIp":
    "DestinationPort":
    "TargetIp": Deprecated, so we gonne ignore it
    "TargetPort": Deprecated, so we gonne ignore it
    "SourceIpInfo": {
      "SourceIpAsnNumber":
      "SourceIpAsnOrgName":
      "SourceIpCountryCode":
      "SourceIpRegion":
      "SourceIpCity"
      "SourceIpPostalCode"
      "SourceIpLatitude"
      "SourceIpLongitude"
      "SourceIpMetroCode"
      "SourceIpAreaCode"
      "SourceIpConnectionType"
    },
    "HttpInfo": {
      "HttpHost": "",
      "HttpRequest": "",
      "HttpMethod": "",
      "HttpReferrer": "",
      "HttpUserAgent": "",
      "HttpVersion": ""
    },
    "CustomInfo": {
      "CustomField1": "",
      "CustomField2": "",
      "CustomField3": "",
      "CustomField4": "",
      "CustomField5": ""
    },
    "Payload": base64 encoded json with meaningful dictionary keys or JSON-string with numbered dictionary keys
  }
  

intelmq.bots.parsers.microsoft.parser_ctip.BOT

alias of MicrosoftCTIPParserBot

class intelmq.bots.parsers.microsoft.parser_ctip.MicrosoftCTIPParserBot(*args, **kwargs)

Bases: ParserBot

Parse JSON data from Microsoft’s CTIP program

overwrite: bool = True

parse(report)

A generator yielding the single elements of the data.

Comments, headers etc. can be processed here. Data needed by self.parse_line can be saved in self.tempdata (list).

Default parser yields stripped lines. Override for your use or use an existing parser, e.g.:

parse = ParserBot.parse_csv

You should do that for recovering lines too.

recover_line = ParserBot.recover_line_csv

parse_azure(line, report)

parse_interflow(line: dict, report)

parse_line(line, report)

A generator which can yield one or more messages contained in line.

Report has the full message, thus you can access some metadata. Override for your use.

Module contents