Skip to main content

Analytic Service

analytic-service is the backend owner of rAInty analytics.

It owns analytics semantics and derived analytics execution. It does not replace data-service as the frontend-facing authorization boundary.

Ownership​

analytic-service owns:

  • analytics domain rules
  • inclusion and exclusion rules for economic facts
  • aggregation logic
  • derived read model generation
  • access to the rainty_analytics schema
  • analytics cache keys and invalidation rules
  • scheduled or event-driven rebuild workflows
  • canonical analytics response shapes for internal service-to-service use

Analytics rules must not be implemented page by page in UserInterface and must not be embedded directly inside data-service.

Boundary with data-service​

data-service remains the only frontend-facing business data boundary for analytics.

data-service must:

  • authenticate frontend requests
  • derive the logged-in user from the token
  • enforce ownership and sharing rules
  • call analytic-service through internal service-to-service APIs
  • return only analytics data the logged-in user is allowed to access

Frontend-facing routes exposed by data-service:

  • GET /data-service/analytics/overview
  • GET /data-service/analytics/properties/:id/summary
  • GET /data-service/analytics/contracts/:id/health
  • GET /data-service/analytics/bank-accounts/:id/summary
  • GET /data-service/analytics/report

These routes live in backend/data-service/routes/analytics.js and delegate to backend/data-service/modules/analytics/analyticsFacade.js.

analytic-service must not trust user identifiers supplied by a browser. User-scoped analytics requests must arrive through data-service, which is responsible for authorization.

Boundary with UserInterface​

UserInterface must call only data-service for analytics data.

UserInterface must not call analytic-service directly, even when the requested data is read-only or already aggregated.

This keeps:

  • authentication behavior centralized
  • ownership filtering consistent
  • frontend deployments independent from internal analytics service topology
  • future replacement of analytic-service possible without changing UserInterface contracts

Contract principle​

analytic-service and data-service must exchange canonical analytics models, not visualization-provider payloads.

The backend contract must stay independent from frontend chart libraries and from any future market analytics engine.

Internal API contract​

analytic-service exposes internal analytics APIs under:

  • POST /internal/analytics/overview
  • POST /internal/analytics/properties/:propertyId/summary
  • POST /internal/analytics/contracts/:contractId/summary
  • POST /internal/analytics/contracts/:contractId/health
  • POST /internal/analytics/bank-accounts/:bankAccountId/summary
  • POST /internal/analytics/report

These endpoints are internal service-to-service APIs. They are not UserInterface APIs.

Authentication requirements:

  • caller must use an internal JWT signed through backend/shared/internalAuth.js
  • token audience must be analytic-service
  • token scope must include analytics:read
  • caller identity must be data-service
  • data-service must receive INTERNAL_JWT_PRIVATE_KEY
  • analytic-service must receive the matching INTERNAL_JWT_PUBLIC_KEY

Request body shape:

{
  "authorizationContext": {
    "authorizedBy": "data-service",
    "requestId": "optional-correlation-id",
    "subject": {
      "userId": 123,
      "partyId": 456
    },
    "accessScope": {
      "propertyIds": [10, 11],
      "contractIds": [20],
      "bankAccountIds": [30],
      "supplierPartyIds": [40]
    }
  },
  "filters": {
    "from": "2026-01-01",
    "to": "2026-12-31"
  },
  "dimensions": ["month", "property"],
  "metrics": ["income", "expenses", "net"]
}

Rules:

  • top-level userId is forbidden
  • authorizationContext.subject.userId must be derived by data-service, never copied from a browser payload
  • accessScope must contain only entity identifiers already authorized by data-service
  • route parameters such as propertyId, contractId, and bankAccountId must be positive integers
  • date filters must use YYYY-MM-DD
  • business-specific analytics execution belongs in dedicated modules, not route handlers

Implementation rule:

  • backend/analytic-service/routes/internalAnalyticsRoutes.js must perform only internal authentication, contract validation, module lookup, and HTTP response/error mapping
  • analytics execution must be delegated to analyticsQueryService
  • route handlers must not contain aggregation rules, inclusion/exclusion decisions, SQL, cache semantics, or chart-provider adaptation
  • adding a new internal analytics endpoint requires adding the contract validation and delegating to a dedicated module method

Canonical response payload​

Every successful analytics response must use the same canonical top-level shape:

{
  "summary": [],
  "facts": [],
  "series": [],
  "breakdown": [],
  "alerts": [],
  "meta": {
    "schemaVersion": "analytics-canonical-v1",
    "generatedAt": "2026-05-13T00:00:00.000Z",
    "currency": "EUR"
  }
}

summary​

summary contains KPI-style values ready for cards or compact totals.

Recommended item shape:

{
  "id": "net_cashflow",
  "label": "Net cashflow",
  "value": 1250.5,
  "unit": "currency",
  "currency": "EUR",
  "period": {
    "from": "2026-01-01",
    "to": "2026-12-31"
  },
  "trend": {
    "direction": "up",
    "delta": 120.25,
    "deltaPercent": 10.6,
    "comparisonPeriod": "previous_period"
  }
}

facts​

facts contains canonical analytics facts after backend inclusion and exclusion rules have been applied.

Recommended item shape:

{
  "id": "fact-123",
  "direction": "INCOME",
  "amount": 1200,
  "currency": "EUR",
  "competenceDate": "2026-01-31",
  "paymentDate": "2026-02-02",
  "propertyId": 10,
  "contractId": 20,
  "bankAccountId": 30,
  "categoryId": 40,
  "source": {
    "type": "bank_transaction_split",
    "id": 500
  }
}

series​

series contains chart-neutral time or category series.

Recommended item shape:

{
  "id": "cashflow_monthly",
  "label": "Cashflow",
  "dimension": "month",
  "measure": "net",
  "points": [
    { "x": "2026-01", "y": 1200 },
    { "x": "2026-02", "y": 900 }
  ]
}

breakdown​

breakdown contains grouped values by dimension.

Recommended item shape:

{
  "dimension": "category",
  "id": "utilities",
  "label": "Utilities",
  "value": 430.2,
  "unit": "currency",
  "currency": "EUR",
  "count": 6,
  "share": 0.24
}

alerts​

alerts contains analytics-derived warnings and actionable states.

Recommended item shape:

{
  "id": "contract-20-overdue",
  "type": "contract_payment_overdue",
  "severity": "warning",
  "messageKey": "analytics.alerts.contractPaymentOverdue",
  "entity": {
    "type": "contract",
    "id": 20
  },
  "dueDate": "2026-02-05",
  "amount": 1200,
  "currency": "EUR"
}

meta​

meta contains response metadata and diagnostics that are safe to expose through data-service.

Recommended fields:

  • schemaVersion
  • generatedAt
  • currency
  • period
  • filters
  • source
  • cache

Vendor independence​

Canonical analytics payloads must not contain chart-provider or analytics-vendor specific configuration.

Forbidden examples:

  • Chart.js datasets
  • react-chartjs-2 component props
  • ECharts options
  • Recharts component models
  • vendor query language responses
  • chartOptions
  • chartType

UserInterface may adapt series and breakdown into chart-library models inside analytics page adapters only.

Economic inclusion and exclusion rules​

analytic-service must build economic facts only from economically validated source records.

Included source records:

  • bank transactions with is_reconciled = 1, reconciliation_status in AUTO_RECONCILED or MANUALLY_RECONCILED, and reconciliation_outcome in VALID, SPLIT, or PARTIAL
  • bank transaction splits whose parent transaction satisfies the same reconciled transaction rule and whose split source is RECONCILIATION_RULE, EXPECTED_MATCH, RENT_MATCH, or MANUAL
  • manual payments that are explicitly CONFIRMED or PAID and linked to an expected expense, utility bill, rent installment, or equivalent payable item
  • expenses created by valid flows and promoted to CONFIRMED or PAID

Excluded source records:

  • draft or pending operational records
  • ignored transactions, even when they are operationally closed
  • unreconciled transactions
  • transactions in AI_SUGGESTED, NEEDS_REVIEW, DUPLICATE, or ERROR
  • transactions with IGNORE, DUPLICATE, or UNKNOWN reconciliation outcomes
  • cancelled, disputed, duplicate, or error operational records

The implementation lives in backend/analytic-service/modules/analytics/inclusionRules.js. economicFactsBuilder.js must reject any fact that is not explicitly marked as economically validated or cannot be validated from its source record.

Persistence rule​

rainty_analytics is a derived read-model schema only.

It may store:

  • canonical economic facts derived from confirmed operational records
  • read models
  • aggregate buckets
  • health projections
  • overdue projections
  • refresh and rebuild logs

It must not store:

  • operational source-of-truth records
  • browser-submitted business data
  • ownership or authorization source tables
  • user settings
  • data that cannot be rebuilt from the rainty operational schema

If a table in rainty_analytics cannot be regenerated from operational data and deterministic analytics rules, it does not belong in rainty_analytics.

Repository layer​

analytic-service uses dedicated repositories for derived analytics persistence.

Current repository files:

  • backend/analytic-service/repositories/economicFactsRepository.js
  • backend/analytic-service/repositories/contractHealthRepository.js
  • backend/analytic-service/repositories/monthlyAggregatesRepository.js

Repository responsibilities:

  • execute parameterized SQL against rainty_analytics
  • upsert derived read model rows
  • list rows by already-authorized user scope
  • delete stale derived rows by deterministic keys

Repository non-responsibilities:

  • deriving analytics rules
  • deciding ownership or sharing
  • accepting browser-provided user identifiers
  • creating operational source-of-truth records
  • returning chart-provider payloads

The repository layer is initialized by backend/analytic-service/modules/main.js and exposed through service.getRepositories().

Business modules​

Analytics business logic lives in dedicated modules under backend/analytic-service/modules/analytics/.

Current modules:

  • economicFactsBuilder.js: builds canonical economic facts from already validated analytics inputs.
  • inclusionRules.js: applies economic inclusion and exclusion rules before facts can be persisted.
  • contractHealthBuilder.js: derives contract health status from scheduled and paid installment data.
  • monthlyAggregationBuilder.js: builds single-dimension monthly aggregate buckets from canonical facts.
  • analyticsQueryService.js: composes repositories into canonical analytics payloads for internal endpoints.

Business module responsibilities:

  • apply deterministic analytics rules
  • prepare derived read models
  • compose canonical payload sections
  • keep chart-provider and vendor-specific models out of backend responses

Business module non-responsibilities:

  • frontend authorization
  • ownership or sharing decisions
  • raw browser request parsing
  • SQL persistence details
  • UI-specific chart adaptation

Current internal route delegation:

  • POST /internal/analytics/overview delegates to analyticsQueryService.getOverview
  • POST /internal/analytics/properties/:propertyId/summary delegates to analyticsQueryService.getPropertySummary
  • POST /internal/analytics/contracts/:contractId/health delegates to analyticsQueryService.getContractHealth
  • POST /internal/analytics/bank-accounts/:bankAccountId/summary delegates to analyticsQueryService.getBankAccountSummary
  • POST /internal/analytics/report delegates to analyticsQueryService.getReport

Refresh and rebuild pipeline​

analytic-service owns refresh and rebuild orchestration for derived analytics read models.

It subscribes to:

  • <ENV>.analytics.refresh

Accepted event shape:

{
  "type": "analytics.refresh.requested",
  "reason": "reconciliation.manual_applied",
  "scope": {
    "type": "TRANSACTION",
    "id": 123
  },
  "impacted": {
    "userId": 2,
    "propertyId": 10,
    "contractId": 20,
    "bankAccountId": 30,
    "supplierPartyId": 40,
    "categoryId": 50,
    "months": ["2026-05"]
  },
  "requestedBy": {
    "service": "reconciliation-service"
  }
}

Initial trigger reasons:

  • reconciliation.auto_applied
  • reconciliation.manual_applied
  • reconciliation.split_changed
  • reconciliation.reset
  • expected_expense.status_changed
  • utility_bill.status_changed
  • rent_installment.status_changed
  • manual_payment.registered
  • analytics.scheduled_partial_rebuild
  • analytics.admin_full_rebuild

Scheduled all-users rebuild:

  • scheduler registers the system job analytics-rebuild-all-users
  • default cron is 0 * * * *, so the job runs once per hour
  • AdminInterface shows it in the Scheduler system jobs page through GET /scheduler/system-jobs
  • runtime settings are ANALYTICS_REBUILD_ALL_USERS_ENABLED, ANALYTICS_REBUILD_ALL_USERS_CRON, and ANALYTICS_REBUILD_ALL_USERS_TIMEZONE
  • the scheduler calls POST /internal/analytics/refresh/all-users
  • the internal token must have audience analytic-service, caller scheduler, and scope analytics:refresh
  • analytic-service then queues one bounded USER refresh event per active non-service user on <ENV>.analytics.refresh

Bounded rebuild scopes:

  • TRANSACTION
  • BANK_TRANSACTION_SPLIT
  • PROPERTY
  • CONTRACT
  • BANK_ACCOUNT
  • EXPECTED_EXPENSE
  • UTILITY_BILL
  • USER

GLOBAL scope is rejected for normal events. It is accepted only for an explicit admin full rebuild command with triggerType: "FULL_REBUILD", adminConfirmed: true, and an admin/DEV execution context.

Refresh planning responsibilities:

  • identify impacted read models: facts, monthly aggregates, contract health, overdue items
  • keep rebuilds bounded to the smallest available entity scope
  • record every refresh run in analytics_refresh_runs
  • expose cache tags that must be invalidated when concrete rebuild workers are attached

Implementation files:

  • backend/analytic-service/modules/analytics/refreshEvents.js
  • backend/analytic-service/modules/analytics/refreshPlanner.js
  • backend/analytic-service/modules/analytics/analyticsRefreshService.js
  • backend/analytic-service/repositories/refreshRunsRepository.js
  • backend/analytic-service/events.manifest.json

Refresh planning now drives concrete workers for fact generation, monthly aggregates, contract health, and targeted cache invalidation. The overdue-item worker is the next read-model worker to attach to the generated plan.

Economic fact generation​

The first concrete rebuild worker generates analytics_economic_facts from operational source tables in the rainty schema.

Implementation files:

  • backend/analytic-service/modules/db/operationalDatabase.js
  • backend/analytic-service/modules/analytics/economicFactsSourceReader.js
  • backend/analytic-service/modules/analytics/economicFactsGenerationService.js
  • backend/analytic-service/modules/analytics/monthlyAggregatesRefreshService.js
  • backend/analytic-service/modules/analytics/contractHealthSourceReader.js
  • backend/analytic-service/modules/analytics/contractHealthRefreshService.js

Operational sources:

  • bank_transaction_splits
  • rent_payment_matches
  • expense_payment_matches
  • utility_bill_payment_matches
  • confirmed or paid manual expenses without a linked bank transaction

Rules:

  • only economically valid reconciled source rows are converted into facts
  • source rows are converted through economicFactsBuilder, so inclusion/exclusion rules still apply before persistence
  • facts are generated per authorized user context derived from property, contract, or bank account relationships
  • split facts take precedence over rent and expected-expense payment matches for the same transaction and payable item, avoiding double counting
  • every bounded refresh deletes facts for the affected scope before inserting regenerated facts
  • every bounded refresh rebuilds monthly aggregate buckets touched by old or new facts
  • GLOBAL rebuild remains admin-only and is not used by normal operational events

Generated fact examples:

  • BANK_TRANSACTION_SPLIT -> income or expense based on the parent transaction direction
  • RENT_PAYMENT_MATCH -> income
  • EXPENSE_PAYMENT_MATCH -> expense
  • UTILITY_BILL_PAYMENT -> expense
  • MANUAL_EXPENSE_PAYMENT -> expense

Monthly aggregates are refreshed from generated facts, not directly from operational tables. The refresh deletes impacted buckets and upserts rebuilt aggregates for these dimensions:

  • PROPERTY
  • CATEGORY
  • BANK_ACCOUNT
  • CONTRACT
  • SERVICE_TYPE

Contract health generation​

analytics_contract_health is rebuilt from operational obligations and actual payments.

Operational sources:

  • rent_installments
  • contract-linked expected_expenses
  • rent_payment_matches
  • expense_payment_matches
  • bank_transaction_splits

Rules:

  • only valid reconciled payments contribute to paid amounts
  • valid splits take precedence over direct rent or expected-expense matches for the same transaction and payable item
  • PAID and MATCHED statuses are used as a fallback only when no payment rows exist
  • cancelled obligations are ignored by contractHealthBuilder
  • rebuilds stay bounded to impacted contracts derived from the refresh scope

Implementation files:

  • backend/analytic-service/modules/analytics/contractHealthSourceReader.js
  • backend/analytic-service/modules/analytics/contractHealthRefreshService.js
  • backend/analytic-service/modules/analytics/contractHealthBuilder.js
  • backend/analytic-service/repositories/contractHealthRepository.js

Operational schema configuration:

  • OPERATIONAL_MYSQL_DATABASE, fallback MYSQL_DATABASE, default rainty
  • OPERATIONAL_MYSQL_HOST, fallback MYSQL_HOST
  • OPERATIONAL_MYSQL_PORT, fallback MYSQL_PORT
  • OPERATIONAL_MYSQL_USER, fallback MYSQL_USER
  • OPERATIONAL_MYSQL_PASSWORD, fallback MYSQL_PASSWORD

Analytics read cache​

analytic-service may use Redis as a read-through cache for common analytics reads after the first complete query path is working.

Redis cache rules:

  • Redis is not source of truth
  • cached payloads must be rebuildable from rainty_analytics
  • cache read or write failures must fall back to repository reads
  • cache keys must include the authorized user and a hash of filters plus authorizationContext.accessScope
  • cached responses must remain canonical analytics payloads
  • cached payloads must not contain chart-provider configuration

Default key patterns:

  • analytics:overview:user:<id>:<scope-hash>
  • analytics:property:<id>:summary:user:<id>:<scope-hash>
  • analytics:contract-health:user:<id>:contract:<id>:<scope-hash>
  • analytics:bank-account:<id>:summary:user:<id>:<scope-hash>
  • analytics:report:user:<id>:<scope-hash>

The implementation lives in backend/analytic-service/modules/analytics/analyticsCache.js. analyticsQueryService owns cache usage because it is the read composition layer. analyticsRefreshService owns invalidation after successful rebuilds.

Invalidation rules:

  • refresh plans emit targeted analytics:* cache tags
  • AnalyticsCache deletes matching Redis keys by prefix because concrete keys include a scope hash
  • invalidation runs after facts, monthly aggregates, or contract health are rebuilt
  • full Redis flushes are not allowed

Configuration:

  • ANALYTICS_CACHE_ENABLED, default true
  • ANALYTICS_CACHE_TTL_SECONDS, default 120

Deployment note​

The current environment-sync-service schema migration flow cannot create the initial rainty_analytics schema by itself because datahub currently rejects CREATE DATABASE, USE, GRANT, and schema-qualified CREATE TABLE statements.

Before TEST can rely on environment-sync-service for this migration, adapt the datahub schema migration runner to support guarded cross-schema additive DDL, or execute the initial schema/grant setup through a DBA/deploy-admin step.

TEST deployment checks:

  1. Verify the analytic-service image is built and published with ANALYTIC_SERVICE_VERSION.
  2. Verify docker-compose.test.yml includes the analytic-service profile and that COMPOSE_PROFILES includes analytic-service.
  3. Verify the TEST environment has both internal JWT secrets configured:
    • INTERNAL_JWT_PRIVATE_KEY is mounted only into data-service.
    • INTERNAL_JWT_PUBLIC_KEY is mounted only into analytic-service.
    • the key pair must match; otherwise analytic-service must reject data-service internal calls.
  4. Verify data-service uses ANALYTIC_SERVICE_URL=http://analytic-service:3013.
  5. Before starting or refreshing analytic-service in TEST, create rainty_analytics and grants with the admin migration step:
CREATE DATABASE IF NOT EXISTS `rainty_analytics`;
GRANT ALL PRIVILEGES ON `rainty_analytics`.* TO 'rainty_user'@'%';
FLUSH PRIVILEGES;
  1. Apply the analytics DDL from db/migrations/001_analytic_service_schema.sql and later additive analytics migrations against TEST.
  2. Verify the schema exists before refresh:
SHOW DATABASES LIKE 'rainty_analytics';
SHOW TABLES FROM `rainty_analytics`;
  1. Verify analytic-service health after deploy:
docker compose --env-file .env -f docker-compose.test.yml -p test ps analytic-service
docker logs --tail=100 test-analytic-service-1
  1. Verify the frontend-facing path end-to-end through data-service, not by calling analytic-service directly from UserInterface.

See also: Analytics Implementation Plan.