Top 10 Use Cases for OpenECoSys in Environmental Monitoring

Getting Started with OpenECoSys — Installation & Setup GuideOpenECoSys is an open, modular ecosystem framework for environmental data collection, processing, and analysis. This guide walks you through installing, configuring, and running OpenECoSys on a typical development or production setup. It covers system requirements, step-by-step installation (local and containerized), basic configuration, connecting sensors and data sources, common troubleshooting, and next steps for customization and deployment.

What you’ll need (prerequisites)

• Operating system: Ubuntu 20.04 LTS or later (instructions also include cross-platform notes for macOS and Windows where relevant).
• CPU & RAM: At least 4 CPU cores and 8 GB RAM for small deployments; 8+ cores and 16+ GB RAM recommended for production.
• Disk space: Minimum 50 GB free (more if you store long-term datasets).
• Docker: Docker Engine (20.10+) and Docker Compose (v2+) for containerized installs.
• Python: Python 3.10+ if running services natively (not in containers).
• Node.js & npm/yarn: Node 16+ for frontend builds.
• Database: PostgreSQL 13+ (or use the bundled containerized DB).
• Hardware interfaces: If connecting physical sensors, appropriate interfaces (USB, serial, or GPIO for single-board computers).
• Network: Ports ⁄₄₄₃ for web, additional internal ports configurable per service.

Choosing an install method

There are three common approaches:

1. Containerized (recommended for most users) — uses Docker Compose or Kubernetes for easy setup and isolation.
2. Native/local (for development or debugging) — run Python services and frontend directly on host.
3. Cloud/Kubernetes (for production scale) — Helm charts or Kubernetes manifests for scalable deployments.

Installation — Containerized (Docker Compose)

This is the fastest, most reproducible method for getting OpenECoSys running.

1. Install Docker and Docker Compose

   • Ubuntu:

     
     sudo apt update sudo apt install -y ca-certificates curl gnupg lsb-release curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg echo  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null sudo apt update sudo apt install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin sudo usermod -aG docker $USER 

   • macOS / Windows: Install Docker Desktop from Docker website.

2. Clone the OpenECoSys repository

   git clone https://github.com/openeconsys/openeconsys.git cd openeconsys 

3. Copy environment templates and set secrets

   cp .env.example .env # Edit .env and set variables like POSTGRES_PASSWORD, REDIS_PASSWORD, SECRET_KEY, etc. 

4. Start services

   docker compose up -d 

   • Monitor logs:

     
     docker compose logs -f 

5. Initialize database (run migrations)

   docker compose exec backend python manage.py migrate docker compose exec backend python manage.py loaddata initial_data.json 

6. Create admin user

   docker compose exec backend python manage.py createsuperuser 

7. Access the app

   • Open http://localhost in your browser (or https:// if TLS configured).
   • Admin UI typically at /admin.

Installation — Native (Development)

Use this method if you plan to develop or debug the code.

1. Install system packages (Ubuntu example)

   sudo apt update sudo apt install -y python3.10 python3.10-venv python3.10-dev build-essential postgresql postgresql-contrib 

2. Create and activate a virtual environment

   python3.10 -m venv venv source venv/bin/activate 

3. Install backend dependencies

   pip install -r requirements.txt 

4. Install frontend dependencies and build

   cd frontend npm install npm run build cd .. 

5. Configure PostgreSQL

   sudo -u postgres createuser --pwprompt ecs_user sudo -u postgres createdb openeconsys_db -O ecs_user # update .env or settings.py with DB credentials 

6. Run migrations and start services

   python manage.py migrate python manage.py runserver 0.0.0.0:8000 

Configuration basics

• .env / config.yml: primary place for secrets and runtime variables (DB URL, Redis, email SMTP, API keys).
• Logging: configure level and destinations in settings; in containers, logs go to Docker logs.
• TLS/HTTPS: For production, terminate TLS at a reverse proxy (nginx, Traefik, or managed LB). Example nginx snippet for proxying to backend:

  
  server {   listen 80;   server_name example.com;   location / {       proxy_pass http://127.0.0.1:8000;       proxy_set_header Host $host;       proxy_set_header X-Real-IP $remote_addr;       proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;       proxy_set_header X-Forwarded-Proto $scheme;   } } 

Connecting sensors and data sources

• Supported protocols: HTTP(S) webhooks, MQTT, LoRaWAN (via network server), Modbus, serial/USB.

• Example: MQTT ingestion

  • Configure broker (Mosquitto or external) and update .env with MQTT_URL and credentials.
  • Sensors publish to topics like sensors/{site}/{device}/telemetry.
  • The ingestion service subscribes and writes to the time-series database.

• LoRaWAN: integrate with The Things Stack or ChirpStack using the network server’s MQTT/HTTP integration.

• Bulk import: CSV/JSON import tools are available in the admin UI and via management commands.

User management and roles

• Built-in roles: Admin, Operator, Analyst, Viewer. Roles map to permissions for creating devices, editing ingest rules, and viewing dashboards.
• Create users via admin UI or CLI (createsuperuser). Assign roles in user profile.

Monitoring, backups, and scaling

• Monitoring: Use Prometheus + Grafana. Exporters for Postgres, system metrics, and application metrics included.
• Backups: schedule PostgreSQL dumps and object storage snapshots (S3-compatible). Example cron for daily DB dump:

  
  0 2 * * * pg_dump -Fc -h db -U ecs_user openeconsys_db > /backups/openeconsys_$(date +%F).dump 

• Scaling: Move to Kubernetes for horizontal scaling. Separate services: ingestion, processing workers, frontend, and long-term storage.

Common issues & troubleshooting

• Containers fail to start: check docker compose logs; ensure environment variables set and ports not in use.
• DB migrations hang: confirm DB reachable, check credentials, and ensure migrations run once (use locks).
• Sensor data not appearing: verify sensor connectivity to broker, check ingestion service logs, confirm topic names.
• High memory usage: inspect worker concurrency settings; reduce worker count or increase swap.

Security best practices

• Keep secrets out of source control; use secret managers (Vault, AWS Secrets Manager) or Docker secrets.
• Run services with least privilege; avoid root in containers.
• Enable HTTPS and HSTS.
• Restrict admin access via IP allowlists or VPN where possible.
• Regularly update dependencies and base images.

Next steps & customization

• Explore plugin architecture to add custom processors or ML models.
• Create dashboards in Grafana and saved queries for recurring analyses.
• Integrate with external systems (ERPs, reporting tools) via REST APIs or webhooks.
• Automate deployments with CI/CD pipelines — GitHub Actions, GitLab CI, or Jenkins.

If you want, I can:

• Provide a ready-to-run docker-compose.yml tuned for a 4-core, 8 GB RAM VM.
• Walk through connecting a specific sensor type (MQTT or LoRaWAN).
• Generate example .env and nginx TLS config.