Configuration and Management ¶

global

systemctl enable redis.service
systemctl start redis.service

apt install supervisor python-pip
pip install supervisor_twiddler

[rpcinterface:twiddler]
supervisor.rpcinterface_factory=supervisor_twiddler.rpcinterface:make_twiddler_rpcinterface

[group:intelmq]

process_manager: supervisor

<bot ID>:
  group: <bot type (Collector, Parser, Expert, Output)>
  name: <human-readable bot name>
  module: <bot code (python module)>
  description: <generic description of the bot>
  parameters:
    <parameter 1>: <value 1>
    <parameter 2>: <value 2>
    <parameter 3>: <value 3>

blocklistde-apache-collector:
  group: Collector
  name: Blocklist.de Apache List
  module: intelmq.bots.collectors.http.collector_http
  description: Blocklist.de Apache Collector fetches all IP addresses which have been reported within the last 48 hours as having run attacks on the service Apache, Apache-DDOS, RFI-Attacks.
  parameters:
    http_url: https://lists.blocklist.de/lists/apache.txt
    name: Blocklist.de Apache
    rate_limit: 3600

blocklistde-apache-collector:
  group: Collector
  name: Blocklist.de Apache List
  module: intelmq.bots.collectors.http.collector_http
  description: Blocklist.de Apache Collector fetches all IP addresses which have been reported within the last 48 hours as having run attacks on the service Apache, Apache-DDOS, RFI-Attacks.
  enabled: false
  parameters:
    http_url: https://lists.blocklist.de/lists/apache.txt
    name: Blocklist.de Apache
    rate_limit: 3600

source-queue: example-bot-queue

destination-queues:
  _default:
    - <first destination pipeline name>
    - <second destination pipeline name>
  _on_error:
    - <optional first destination pipeline name in case of errors>
    - <optional second destination pipeline name in case of errors>
  other-path:
    - <second destination pipeline name>
    - <third destination pipeline name>

{
    "<message type>": {
        "<field 1>": {
            "description": "<field 1 description>",
            "type": "<field value type>"
        },
        "<field 2>": {
            "description": "<field 2 description>",
            "type": "<field value type>"
        }
    },
}

{
    "event": {
        "destination.asn": {
            "description": "The autonomous system number from which originated the connection.",
            "type": "Integer"
        },
        "destination.geolocation.cc": {
            "description": "Country-Code according to ISO3166-1 alpha-2 for the destination IP.",
            "regex": "^[a-zA-Z0-9]{2}$",
            "type": "String"
        },
    },
}

blocklistde-apache-collector:
  name: Generic URL Fetcher
  group: Collector
  module: intelmq.bots.collectors.http.collector_http
  description: All IP addresses which have been reported within the last 48 hours as having run attacks on the service Apache, Apache-DDOS, RFI-Attacks.
  enabled: false
  run_mode: scheduled
  parameters:
    feed: Blocklist.de Apache
    provider: Blocklist.de
    http_url: https://lists.blocklist.de/lists/apache.txt
    ssl_client_certificate: null

0 0 * * * intelmqctl start blocklistde-apache-collector

blocklistde-apache-parser:
  name: Blocklist.de Parser
  group: Parser
  module: intelmq.bots.parsers.blocklistde.parser
  description: Blocklist.DE Parser is the bot responsible to parse the report and sanitize the information.
  enabled: false
  run_mode: continuous
  parameters: ...

intelmqctl start blocklistde-apache-parser

redis-cli FLUSHDB
redis-cli FLUSHALL

intelmqdump -h
usage:
    intelmqdump [botid]
    intelmqdump [-h|--help]

intelmqdump can inspect dumped messages, show, delete or reinject them into
the pipeline. It's an interactive tool, directly start it to get a list of
available dumps or call it with a known bot id as parameter.

positional arguments:
  botid       botid to inspect dumps of

optional arguments:
  -h, --help  show this help message and exit
  --truncate TRUNCATE, -t TRUNCATE
                        Truncate raw-data with more characters than given. 0 for no truncating. Default: 1000.

Interactive actions after a file has been selected:
- r, Recover by IDs
  > r id{,id} [queue name]
  > r 3,4,6
  > r 3,7,90 modify-expert-queue
  The messages identified by a consecutive numbering will be stored in the
  original queue or the given one and removed from the file.
- a, Recover all
  > a [queue name]
  > a
  > a modify-expert-queue
  All messages in the opened file will be recovered to the stored or given
  queue and removed from the file.
- d, Delete entries by IDs
  > d id{,id}
  > d 3,5
  The entries will be deleted from the dump file.
- d, Delete file
  > d
  Delete the opened file as a whole.
- s, Show by IDs
  > s id{,id}
  > s 0,4,5
  Show the selected IP in a readable format. It's still a raw format from
  repr, but with newlines for message and traceback.
- e, Edit by ID
  > e id
  > e 0
  > e 1,2
  Opens an editor (by calling `sensible-editor`) on the message. The modified message is then saved in the dump.
- q, Quit
  > q

$ intelmqdump
 id: name (bot id)                    content
  0: alienvault-otx-parser            1 dumps
  1: cymru-whois-expert               8 dumps
  2: deduplicator-expert              2 dumps
  3: dragon-research-group-ssh-parser 2 dumps
  4: file-output2                     1 dumps
  5: fraunhofer-dga-parser            1 dumps
  6: spamhaus-cert-parser             4 dumps
  7: test-bot                         2 dumps
Which dump file to process (id or name)? 3
Processing dragon-research-group-ssh-parser: 2 dumps
  0: 2015-09-03T13:13:22.159014 InvalidValue: invalid value u'NA' (<type 'unicode'>) for key u'source.asn'
  1: 2015-09-01T14:40:20.973743 InvalidValue: invalid value u'NA' (<type 'unicode'>) for key u'source.asn'
(r)ecover by ids, recover (a)ll, delete (e)ntries, (d)elete file, (s)how by ids, (q)uit, edit id (v)? d
Deleted file /opt/intelmq/var/log/dragon-research-group-ssh-parser.dump

tail -f /opt/intelmq/var/log/*.log
tail -f /var/log/intelmq/*.log

pip3 uninstall intelmq
rm -r /opt/intelmq

Configuration and Management ¶

Configure services ¶

Configuration ¶

/opt and LSB paths ¶

Overview ¶

Systemwide Configuration (global)¶

Logging ¶

Log rotation ¶

Error Handling ¶

Miscellaneous ¶

Using supervisor as process manager (Beta)¶

Runtime Configuration ¶

Pipeline Configuration ¶

Source queue ¶

Destination queues ¶

AMQP (Beta)¶

Multithreading (Beta)¶

Harmonization Configuration ¶

Utilities ¶

Management ¶

Web interface: IntelMQ Manager ¶

Command-line interface: intelmqctl ¶

Botnet Concept ¶

Scheduled Run Mode ¶

Continuous Run Mode ¶

Reloading ¶

Forcing reset pipeline and cache (be careful)¶

Error Handling ¶

Tool: intelmqdump ¶

Monitoring Logs ¶

Uninstall ¶

Integration with ticket systems, etc.¶

Frequently Asked Questions ¶

Additional Information ¶

Bash Completion ¶

Navigation

Related Topics