Welcome to Spidermon’s documentation!

Spidermon is a framework to build monitors for Scrapy spiders. It offers the following features:

• It can check the output data produced by Scrapy (or other sources) and verify it against a schema or model that defines the expected structure, data types and value restrictions. It supports data validation based on the jsonschema library (https://github.com/Julian/jsonschema).

• It allows you to define conditions that should trigger an alert based on Scrapy stats.

• It supports notifications via email, Slack, Telegram and Discord.

• It can generate custom reports.

Contents

• Installation
• Getting started
  • Our sample project
  • Enabling Spidermon
  • Our first monitor
  • Notifications
  • Slack notifications
  • Telegram notifications
  • Discord notifications
  • Item validation
• Monitoring your jobs
  • Monitors
  • Monitor Suites
  • Base Stat Monitor
  • The Basic Monitors
  • Is there a Basic Scrapy Suite ready to use?
  • Periodic Monitors
  • What to monitor?
• Item Validation
  • Using JSON Schema
  • Settings
  • Validation in Monitors
• Expression Monitors
  • How to create an expression monitor
  • Expression monitor schema
• Settings
  • Built-in settings reference
• “How-to” guides
  • How do I add required fields coverage validation?
  • How do I configure a Slack bot for Spidermon?
  • How do I configure a Telegram bot for Spidermon?
  • How do I configure a Discord bot for Spidermon?
  • How do I compare Spider executions?
• Actions
  • Email action
  • Slack action
  • Telegram action
  • Discord action
  • Job tags action
  • File Report Action
  • S3 Report action
  • Sentry action
  • SNS action
  • Custom Actions
  • Fallback Actions
• Release notes
  • 1.22.0 (2024-05-08)
  • 1.21.0 (2024-04-18)
  • 1.20.0 (2023-09-01)
  • 1.19.0 (2023-07-07)
  • 1.18.0 (2023-03-13)
  • 1.17.1 (2023-01-05)
  • 1.17.0 (2022-09-12)
  • 1.16.2 (2021-12-23)
  • 1.15.2 (2021-10-04)
  • 1.15.1 (2021-10-04)
  • 1.15.0 (2021-04-06)
  • 1.14.0 (2020-10-05)
  • 1.13.0 (2020-06-23)
  • 1.12.2 (2020-05-07)
  • 1.12.1 (2020-05-07)
  • 1.12.0 (2020-01-09)
  • 1.11.0 (2019-08-02)
  • 1.10.2 (2019-07-01)
  • 1.10.1 (2019-07-01)
  • 1.10.0 (2019-06-12)
  • 1.9.0 (2019-03-11)
  • 1.8.0 (2019-01-08)
  • 1.7.0 (2018-12-04)
  • 1.6.0 (2018-11-09)
  • 1.5.0 (2018-09-19)
  • 1.4.0 (2018-08-17)
  • 1.3.0 (2018-08-02)
  • 1.2.0 (2018-04-04)
  • 1.1.0 (2018-03-23)
  • 1.0.0 (2018-03-08)