Skip to main content

MongoDB

Overview​

MongoDB is a popular NoSQL database program that uses JSON-like documents with optional schemas. SaddleData supports MongoDB as both a source and a destination, allowing you to move data in and out of your document collections seamlessly.

Prerequisites​

Before connecting to MongoDB, please ensure you have the following:

  • A dedicated user for your database with appropriate permissions (read for source, write for destination).
  • Network access allowed for SaddleData's IP addresses if you are using the Cloud service, or connectivity from your Worker if using the Remote Agent.

Configuration​

When creating a MongoDB Integration, you will need to provide the following information:

  • Connection URI: The standard MongoDB connection string (e.g., mongodb+srv://user:[email protected]/...).
  • Database Name: The name of the database you want to access.

Supported Capabilities​

MongoDB as a Source​

  • Discovery: Automatically discovers collections in the specified database.
  • Schema Inference: Infers the schema (columns and types) by sampling documents from the collection.
  • Sync Modes:
    • Full Refresh: Reads all documents from the collection.
    • Incremental: Reads new documents based on a cursor field (e.g., _id or a timestamp field).

MongoDB as a Destination​

  • Writing: Writes records as documents into the specified collection.
  • Sync Modes:
    • Full Refresh - Overwrite: Replaces the collection with new data.
    • Incremental - Append: Inserts new documents.
    • Incremental - Deduped (Upsert): Updates existing documents based on the _id field or a specified primary key.

Schema Evolution​

SaddleData detects schema changes in your MongoDB collections. New fields found in documents can be automatically added to the downstream schema definition, and mapped to destination columns.

Declarative Configuration​

apiVersion: v1
kind: Connection
metadata:
  name: mongodb-connection
spec:
  connectorId: mongodb
  configuration:
    uri: mongodb+srv://...
    database: my_database
    capability: both