Skip to main content

Analytics Implementation Plan

This document defines how the analytics layer must be designed and implemented in rAInty.

It is intended as the input for phase 2 implementation.

The goal is to build a single, coherent, reusable analytics model that:

  • uses only secure user-scoped data
  • computes economic results only from reconciled transactions
  • supports multiple pages and visualizations
  • avoids duplicated calculation logic across frontend pages
  • remains scalable as users, transactions, and entities grow

1. Scope and principles​

1.1 Functional scope​

The analytics layer must support:

  • Overview page KPIs
  • contract payment health indicators
  • bank account economic summary
  • property economic summary
  • detailed property expense breakdown
  • cross-dimensional income/expense reporting
  • overdue and disputed item monitoring

1.2 Data inclusion rules​

Economic analytics must include only data that is economically validated.

Included:

  • bank transactions with is_reconciled = 1, reconciliation_status in AUTO_RECONCILED or MANUALLY_RECONCILED, and reconciliation_outcome in VALID, SPLIT, or PARTIAL
  • transaction splits whose parent transaction satisfies the same reconciled transaction rule and whose split source is RECONCILIATION_RULE, EXPECTED_MATCH, RENT_MATCH, or MANUAL
  • manual payments that are explicitly CONFIRMED or PAID and linked to an expected expense, utility bill, rent installment, or equivalent payable item
  • expense records created by valid flows and promoted to CONFIRMED or PAID

Excluded:

  • ignored transactions, even when operational reconciliation marks them as closed
  • non reconciled transactions
  • transactions with reconciliation_status in UNRECONCILED, IGNORED, AI_SUGGESTED, NEEDS_REVIEW, DUPLICATE, or ERROR
  • transactions with reconciliation_outcome in IGNORE, DUPLICATE, or UNKNOWN
  • draft, pending, ignored, cancelled, disputed, duplicate, or error operational items

The implementation owner for these rules is backend/analytic-service/modules/analytics/inclusionRules.js. economicFactsBuilder.js must reject any canonical economic fact that does not carry a valid economic validation result or cannot be validated from its source record.

1.3 Core architectural principles​

  • data-service remains the security boundary for frontend access
  • business rules stay in backend
  • frontend may perform local visual aggregations only on already canonical datasets
  • analytics must not be reimplemented page by page
  • pages must consume analytics APIs and shared transform modules

2. Conceptual separation of datasets​

The analytics model must separate three concepts.

2.1 Economic actuals​

This is the real confirmed economic impact.

Examples:

  • rent payment received
  • utility bill paid
  • condominium installment paid
  • manually confirmed expense

This dataset is the basis for:

  • income
  • expense
  • net cashflow
  • breakdowns by dimension
  • reports

2.2 Expected obligations​

This is the operational layer of what should happen.

Examples:

  • expected rent installments
  • expected expenses
  • utility bills due

This dataset is the basis for:

  • overdue items
  • pending items
  • contract health
  • operational alerts

2.3 Health indicators​

This is a derived status layer built from expected obligations plus actual payments.

Examples:

  • contract payment status: green / yellow / red
  • expected expense on time / late / overdue
  • utility bill received / overdue / disputed / paid

This layer is not a source table. It is a derived view or read model.

3. Backend ownership model​

3.1 Security and ownership​

All frontend-facing analytics access must pass through data-service.

UserInterface must call only data-service for analytics data and must never call analytic-service directly.

data-service is responsible for:

  • deriving the logged-in user
  • filtering by ownership and sharing rules
  • exposing only authorized data
  • returning canonical analytics datasets

If data-service returns data outside the user scope, that is an architectural and security defect.

Frontend-facing analytics routes:

  • GET /data-service/analytics/overview
  • GET /data-service/analytics/properties/:id/summary
  • GET /data-service/analytics/contracts/:id/summary
  • GET /data-service/analytics/contracts/:id/health
  • GET /data-service/analytics/bank-accounts/:id/summary
  • GET /data-service/analytics/report

Implementation files:

  • backend/data-service/routes/analytics.js
  • backend/data-service/modules/analytics/analyticsFacade.js

The route file must remain thin. The facade module derives the trusted user context from req.userCtx, applies ownership/sharing checks, builds authorizationContext.accessScope, signs the internal service JWT, calls analytic-service, and returns only the canonical payload authorized for the caller.

3.2 Analytics implementation ownership​

Analytics must not be implemented inside data-service.

The analytics capability must be owned by a dedicated backend microservice named analytic-service.

This service owns:

  • analytics domain rules
  • derived analytics schema access
  • read model generation
  • aggregation logic
  • cache keys and cache invalidation rules
  • scheduled or event-driven analytics rebuilds
  • canonical analytics response shapes

data-service remains the frontend-facing security boundary. It must:

  • authenticate the request
  • derive the logged-in user from the token
  • enforce ownership and sharing filters before exposing results
  • call analytic-service through an internal service-to-service API
  • return only analytics data the logged-in user is allowed to access

analytic-service must not trust user identifiers supplied by the browser. When it receives a user-scoped request from data-service, it must treat data-service as the authorization boundary and still validate all required internal parameters.

Internal requests from data-service to analytic-service must use internal service-to-service authentication and must carry an authorizationContext produced by data-service.

The request body must not contain a top-level userId. If a user identifier is needed for cache scope or analytics filtering, it must appear only under authorizationContext.subject.userId after data-service has derived it from the authentication token.

Initial internal endpoints:

  • POST /internal/analytics/overview
  • POST /internal/analytics/properties/:propertyId/summary
  • POST /internal/analytics/contracts/:contractId/health
  • POST /internal/analytics/bank-accounts/:bankAccountId/summary
  • POST /internal/analytics/report

These endpoints must stay thin. They may perform internal authentication, contract validation, module lookup, and HTTP response/error mapping only. Analytics execution must be delegated to analyticsQueryService or another dedicated analytics module. Routes must not contain aggregation rules, inclusion/exclusion decisions, SQL, cache semantics, or chart-provider adaptation.

Implementation structure:

  • backend/analytic-service/
  • backend/analytic-service/modules/
  • backend/analytic-service/routes/
  • backend/analytic-service/repositories/
  • backend/analytic-service/docs/

Initial repository files:

  • backend/analytic-service/repositories/economicFactsRepository.js
  • backend/analytic-service/repositories/contractHealthRepository.js
  • backend/analytic-service/repositories/monthlyAggregatesRepository.js

Repositories must own SQL persistence only. Analytics rules, rebuild logic, ownership decisions, and payload shaping must remain in dedicated service modules.

Initial business modules:

  • backend/analytic-service/modules/analytics/economicFactsBuilder.js
  • backend/analytic-service/modules/analytics/inclusionRules.js
  • backend/analytic-service/modules/analytics/contractHealthBuilder.js
  • backend/analytic-service/modules/analytics/monthlyAggregationBuilder.js
  • backend/analytic-service/modules/analytics/analyticsQueryService.js

Business modules own deterministic analytics rules and canonical payload composition. They must not own frontend authorization, SQL persistence details, or chart-provider adaptation.

The microservice must follow the standard backend service rules:

  • start from backend/__TemplateService
  • use BaseService
  • use serverFactory
  • use shared logger
  • expose standard health and status routes
  • read configurable parameters through getConfigValue
  • document APIs, events, configuration, and ownership in the help documentation

4. Persistence model​

4.1 Dedicated analytics schema​

Analytics tables should live in a dedicated MySQL schema on the same server, not mixed with the operational schema.

Recommended schemas:

  • rainty for operational data
  • rainty_analytics for derived analytics data

Reasons:

  • clear separation between source of truth and derived data
  • easier rebuild and maintenance
  • safer migration strategy
  • simpler backup and recovery by scope

4.2 Source of truth vs derived data​

Source of truth stays in operational tables such as:

  • bank_transactions
  • bank_transaction_splits
  • rent_payment_matches
  • expense_payment_matches
  • expenses
  • expected_expenses
  • utility_bills
  • service_contracts
  • contracts
  • properties
  • parties

Derived analytics data lives in rainty_analytics.

Redis is not the source of truth. Redis is only a read cache.

rainty_analytics must contain only derived data and read models. It must not contain operational source-of-truth records.

Allowed content:

  • facts derived from confirmed operational records
  • read models rebuilt from operational tables
  • aggregate buckets
  • health status projections
  • overdue item projections
  • refresh and rebuild execution logs

Forbidden content:

  • browser-submitted business data
  • contracts, properties, parties, bank transactions, expenses, or documents as primary records
  • authorization or ownership source tables
  • workflow state that cannot be rebuilt from rainty
  • user settings or user preferences
  • chart-provider configuration

If rainty_analytics is lost, the platform must be able to rebuild it from rainty plus deterministic analytics rules. Any analytics table that cannot be rebuilt is a design defect.

5. Analytics data model​

5.1 Canonical fact model​

The foundation is a canonical fact dataset.

Suggested base table:

  • rainty_analytics.analytics_economic_facts

Each row represents a single economically confirmed fact.

Suggested fields:

  • id
  • user_id
  • party_id
  • source_type
  • source_id
  • bank_transaction_id
  • bank_transaction_split_id
  • property_id
  • contract_id
  • service_contract_id
  • supplier_party_id
  • tenant_party_id
  • bank_account_id
  • category_id
  • service_type
  • economic_direction (INCOME, EXPENSE)
  • amount
  • currency
  • recognized_at
  • payment_date
  • competence_date
  • competence_month
  • competence_quarter
  • payment_channel (BANK, MANUAL)
  • document_id
  • reconciliation_run_id
  • metadata_json
  • created_at
  • updated_at

5.2 Source types​

Suggested source_type values:

  • RENT_PAYMENT_MATCH
  • EXPENSE_PAYMENT_MATCH
  • BANK_TRANSACTION_SPLIT
  • MANUAL_EXPENSE_PAYMENT
  • UTILITY_BILL_PAYMENT
  • EXPECTED_EXPENSE_PAYMENT

The implementation may evolve, but the principle is fixed:

  • each fact row must have one unambiguous source and one economic meaning

5.3 Health and operational datasets​

Suggested derived tables:

  • rainty_analytics.analytics_contract_health
  • rainty_analytics.analytics_overdue_items

Suggested contract health fields:

  • user_id
  • contract_id
  • property_id
  • status (GREEN, YELLOW, RED)
  • scheduled_count
  • paid_count
  • overdue_count
  • first_open_due_date
  • max_delay_days
  • grace_days_used
  • updated_at

Suggested overdue item fields:

  • user_id
  • item_type (EXPECTED_EXPENSE, UTILITY_BILL, RENT_INSTALLMENT)
  • item_id
  • property_id
  • contract_id
  • supplier_party_id
  • category_id
  • due_date
  • amount
  • status
  • severity
  • updated_at

5.4 Aggregated datasets​

Suggested first aggregate table:

  • rainty_analytics.analytics_monthly_aggregates

Suggested fields:

  • user_id
  • period_month
  • dimension_type
  • dimension_id
  • dimension_label
  • income_total
  • expense_total
  • net_total
  • facts_count
  • updated_at

Dimension types:

  • PROPERTY
  • SUPPLIER
  • CATEGORY
  • BANK_ACCOUNT
  • CONTRACT
  • SERVICE_TYPE

This table must not attempt to materialize every possible multi-dimensional cube combination.

Start with:

  • monthly aggregates by single dimension
  • derive more complex combinations from canonical facts when needed

6. Refresh and recalculation strategy​

6.1 Recalculation principle​

Never rebuild the analytics of the entire platform after every change.

Use delta-based updates:

  • update only the facts impacted by the business event
  • update only the aggregate buckets touched by those facts
  • invalidate only the related cache keys

6.2 Events that trigger recalculation​

Analytics refresh must be triggered by:

  • automatic reconciliation applied
  • manual reconciliation applied
  • reconciliation split created or edited
  • reconciliation reset
  • expected expense status changes
  • utility bill status changes
  • rent installment status changes
  • manual payment registration

Initial Redis event channel:

  • <ENV>.analytics.refresh

Event type:

  • analytics.refresh.requested

Initial reason values:

  • reconciliation.auto_applied
  • reconciliation.manual_applied
  • reconciliation.split_changed
  • reconciliation.reset
  • expected_expense.status_changed
  • utility_bill.status_changed
  • rent_installment.status_changed
  • manual_payment.registered
  • analytics.scheduled_partial_rebuild
  • analytics.admin_full_rebuild

Recommended event payload:

{
  "type": "analytics.refresh.requested",
  "reason": "reconciliation.manual_applied",
  "scope": {
    "type": "TRANSACTION",
    "id": 123
  },
  "impacted": {
    "userId": 2,
    "propertyId": 10,
    "contractId": 20,
    "bankAccountId": 30,
    "months": ["2026-05"]
  },
  "requestedBy": {
    "service": "reconciliation-service"
  }
}

6.3 Recalculation levels​

Incremental update​

Default path.

Update:

  • affected rows in analytics_economic_facts
  • affected monthly aggregate buckets
  • affected contract health rows
  • affected overdue rows

Scheduled partial rebuild​

Nightly or periodic.

Recompute:

  • recent months
  • recently touched users
  • recently touched entities

Goal:

  • fix drift
  • catch missed events
  • keep cost bounded

Full rebuild​

Admin-only or DEV-only maintenance path.

Use only for:

  • model changes
  • bug recovery
  • historical rebuild
  • large migration

GLOBAL scope must be rejected for normal events. It is allowed only for an explicit command with:

  • triggerType: "FULL_REBUILD"
  • adminConfirmed: true
  • admin execution context, or DEV/LOCAL environment

6.4 Delta update keying​

Every business event should provide enough information to narrow the recalculation scope.

Typical impacted keys:

  • user_id
  • property_id
  • contract_id
  • supplier_party_id
  • bank_account_id
  • category_id
  • affected month(s)

The refresh pipeline must operate on those keys, not on the full dataset.

Initial bounded scopes:

  • TRANSACTION
  • BANK_TRANSACTION_SPLIT
  • PROPERTY
  • CONTRACT
  • BANK_ACCOUNT
  • EXPECTED_EXPENSE
  • UTILITY_BILL
  • USER

Implementation files:

  • backend/analytic-service/modules/analytics/refreshEvents.js
  • backend/analytic-service/modules/analytics/refreshPlanner.js
  • backend/analytic-service/modules/analytics/analyticsRefreshService.js
  • backend/analytic-service/repositories/refreshRunsRepository.js
  • backend/analytic-service/events.manifest.json

The first refresh implementation records the run and creates a bounded plan. The concrete source readers and read-model rebuild workers are the next implementation step.

6.5 Economic fact generation worker​

The first concrete refresh worker generates analytics_economic_facts.

Operational sources:

  • bank_transaction_splits
  • rent_payment_matches
  • expense_payment_matches
  • utility_bill_payment_matches
  • confirmed or paid manual expenses without a linked bank transaction

Generation behavior:

  • read operational data from the rainty schema
  • apply the inclusion/exclusion rules through economicFactsBuilder
  • fan out generated facts per authorized user context derived from property, contract, or bank account relationships
  • delete existing facts for the bounded refresh scope before upserting regenerated facts
  • prefer split facts over rent or expected-expense match facts for the same transaction and payable item to avoid double counting

Initial supported refresh scopes for fact generation:

  • TRANSACTION
  • BANK_TRANSACTION_SPLIT
  • PROPERTY
  • CONTRACT
  • BANK_ACCOUNT
  • EXPECTED_EXPENSE
  • UTILITY_BILL
  • USER

Implementation files:

  • backend/analytic-service/modules/db/operationalDatabase.js
  • backend/analytic-service/modules/analytics/economicFactsSourceReader.js
  • backend/analytic-service/modules/analytics/economicFactsGenerationService.js
  • backend/analytic-service/modules/analytics/monthlyAggregatesRefreshService.js
  • backend/analytic-service/modules/analytics/contractHealthSourceReader.js
  • backend/analytic-service/modules/analytics/contractHealthRefreshService.js

6.6 Monthly aggregate refresh worker​

Monthly aggregates are derived from analytics_economic_facts.

After every bounded fact rebuild:

  • collect buckets touched by facts that existed before the rebuild
  • collect buckets touched by facts generated by the rebuild
  • delete those buckets from analytics_monthly_aggregates
  • rebuild aggregates from the current generated facts

Initial dimensions:

  • PROPERTY
  • CATEGORY
  • BANK_ACCOUNT
  • CONTRACT
  • SERVICE_TYPE

This keeps aggregates consistent with facts and avoids a global aggregate rebuild after every operational event.

6.7 Contract health refresh worker​

Contract health is derived from rental contract obligations and actual payment evidence.

Operational obligations:

  • rent_installments
  • contract-linked expected_expenses

Operational payment evidence:

  • rent_payment_matches
  • expense_payment_matches
  • bank_transaction_splits

Refresh rules:

  • only reconciled, economically valid bank transactions are counted as payments
  • valid splits win over direct payment matches for the same payable item
  • PAID and MATCHED source statuses are used as fallback only when no payment rows exist
  • cancelled obligations are ignored
  • rebuild deletes and rewrites only the impacted contract health rows

Implementation files:

  • backend/analytic-service/modules/analytics/contractHealthSourceReader.js
  • backend/analytic-service/modules/analytics/contractHealthRefreshService.js
  • backend/analytic-service/modules/analytics/contractHealthBuilder.js
  • backend/analytic-service/repositories/contractHealthRepository.js

7. Caching strategy​

7.1 Redis role​

Redis is used only as a read cache.

Do not store analytics truth only in Redis.

7.2 Good cache candidates​

  • overview summary
  • overview alerts
  • property summary
  • bank account summary
  • contract health lists
  • common report payloads with repeated filters

7.3 Cache key examples​

  • analytics:overview:user:2:<scope-hash>
  • analytics:property:15:summary:user:2:<scope-hash>
  • analytics:bank-account:7:summary:user:2:<scope-hash>
  • analytics:contract-health:user:2:contract:9:<scope-hash>
  • analytics:report:user:2:<filters-hash>

7.4 Cache invalidation​

Invalidate only affected keys when facts, monthly aggregates, or health models change.

Do not clear the entire analytics cache.

Implementation rules:

  • refresh planning emits targeted analytics:* cache tags
  • concrete keys include filter/access-scope hashes, so invalidation deletes by analytics prefix/tag
  • only keys beginning with analytics: may be invalidated by this worker
  • cache invalidation runs after the rebuild workers complete successfully

8. Backend API strategy​

8.1 API style​

The backend must expose analytics-ready datasets, not raw operational tables.

The frontend should not reconstruct economic truth from:

  • bank transactions
  • matches
  • utility bills
  • expected expenses

The backend must return a canonical analytics payload that is:

  • secure
  • user-scoped
  • economically correct
  • consistent across pages

8.2 First API set​

Recommended first endpoints:

  • GET /data-service/analytics/overview
  • GET /data-service/analytics/properties/:id/summary
  • GET /data-service/analytics/contracts/:id/health
  • GET /data-service/analytics/bank-accounts/:id/summary
  • GET /data-service/analytics/report

8.3 API responsibilities​

Backend APIs must handle:

  • user security filtering
  • business inclusion rules
  • canonical row building
  • optional pre-aggregations
  • user settings application where business-critical

8.4 Frontend API contract style​

Recommended payload style:

  • summary
  • facts
  • series
  • breakdown
  • alerts
  • meta

This allows one fetch per page and multiple local visual aggregations in frontend.

All analytics responses must use this canonical top-level shape.

{
  "summary": [],
  "facts": [],
  "series": [],
  "breakdown": [],
  "alerts": [],
  "meta": {
    "schemaVersion": "analytics-canonical-v1",
    "generatedAt": "2026-05-13T00:00:00.000Z",
    "currency": "EUR"
  }
}

Section responsibilities:

  • summary: KPI-style metrics and totals
  • facts: canonical economic facts after backend inclusion and exclusion rules
  • series: chart-neutral time or category series
  • breakdown: grouped values by dimension
  • alerts: analytics-derived warnings and actionable states
  • meta: schema version, filters, period, source, cache and safe diagnostics

Payload rules:

  • backend responses must not contain Chart.js datasets or options
  • backend responses must not contain react-chartjs-2 props
  • backend responses must not contain vendor-specific chart or market analytics engine payloads
  • chart-provider adaptation belongs only in UserInterface analytics page adapters
  • external analytics engine output must be normalized into this canonical shape before reaching UserInterface

9. Frontend architecture​

9.1 Core principle​

Frontend may aggregate locally only after backend has returned a secure and canonical analytics dataset.

Frontend must not rebuild domain rules.

9.2 Separation of responsibilities​

API layer​

Files dedicated to analytics fetches.

Suggested:

  • frontend/UserInterface/src/api/dataService/analytics.ts

Responsibilities:

  • fetch only
  • no business calculations

Analytics transform layer​

This is the key reusable layer.

Suggested:

  • frontend/UserInterface/src/features/analytics/

Example structure:

  • types.ts
  • dimensions.ts
  • filters.ts
  • transforms/
  • hooks/

This layer must be designed as a reusable analytics engine, not as page-specific helpers.

Hooks layer​

Suggested:

  • useOverviewAnalytics.ts
  • usePropertyAnalytics.ts
  • useIncomeExpenseReport.ts

Responsibilities:

  • call APIs
  • apply reusable transforms
  • expose view-ready data to pages

Presentation layer​

Pages and components must only:

  • choose filters
  • call hooks
  • render data

Pages must not contain local business aggregation logic.

10. Frontend analytics transform layer design​

10.1 Goal​

The transform layer must be highly reusable because:

  • dimensions are finite
  • measures are finite
  • views are many

It should behave like a lightweight local analytics engine over a canonical dataset.

10.2 Canonical input model​

All frontend transforms should start from a single canonical type, for example:

  • AnalyticsFact

Suggested fields:

  • propertyId
  • propertyName
  • contractId
  • contractName
  • supplierId
  • supplierName
  • categoryId
  • categoryName
  • bankAccountId
  • bankAccountName
  • serviceType
  • direction
  • amount
  • competenceDate
  • paymentDate
  • status

10.3 Dimension registry​

Dimensions should be centrally registered instead of handled with repeated if and switch.

Suggested dimensions:

  • property
  • supplier
  • category
  • bankAccount
  • contract
  • serviceType
  • month
  • quarter

The registry should define:

  • id selector
  • label selector
  • bucket strategy where applicable

10.4 Primitive transform operations​

The transform layer should provide generic primitives such as:

  • filter facts
  • group facts by dimension
  • aggregate facts by metric
  • build time series
  • rank groups
  • compute totals

The layer must be query-oriented, not page-oriented.

10.5 Output model standards​

To maximize UI reuse, transforms should output a small set of standard view models.

Suggested outputs:

  • SummaryMetric[]
  • BreakdownRow[]
  • TimeSeriesPoint[]
  • HealthRow[]
  • RankingRow[]

10.6 Analytics view adapters​

On top of generic primitives, create adapters that map generic analytics outputs into UI-specific models.

Examples:

  • overview KPI adapter
  • property summary adapter
  • supplier ranking adapter
  • expense category chart adapter
  • contract health adapter

These adapters must remain inside the analytics feature or the analytics pages that use them. They are not a global application abstraction and must not leak chart-library-specific models into shared business APIs.

10.7 What must stay out of pages​

Pages must not contain:

  • reduce(...) business logic
  • groupBy(...) business logic
  • local cashflow derivation logic
  • local health color rules

Those belong to the transform layer or to backend if business-critical.

10.8 Chart provider strategy​

The Phase 1 chart provider for UserInterface analytics is Chart.js through react-chartjs-2.

This is an implementation choice for rendering charts in the frontend, not part of the analytics API contract.

Rules:

  • analytic-service must return canonical analytics data, not Chart.js datasets or options
  • data-service must not expose chart-provider-specific payloads
  • analytics pages may adapt canonical view models into Chart.js models locally
  • chart adapters must stay inside analytics-specific frontend code
  • shared UI components must receive rAInty-owned view models, not Chart.js objects
  • pages outside analytics must not depend on analytics chart adapters

Recommended frontend shape:

  • frontend/UserInterface/src/features/analytics/types.ts
  • frontend/UserInterface/src/features/analytics/transforms/
  • frontend/UserInterface/src/components/pages/analytics/adapters/chartJsAdapter.ts
  • frontend/UserInterface/src/components/pages/analytics/components/AnalyticsLineChart.tsx
  • frontend/UserInterface/src/components/pages/analytics/components/AnalyticsBreakdownChart.tsx

The wrappers may use react-chartjs-2 internally, but analytics pages should render rAInty-owned components such as AnalyticsLineChart or AnalyticsBreakdownChart instead of importing Line, Doughnut, or other provider components directly.

Provider replacement rule:

If rAInty replaces Chart.js with another JavaScript chart library, only the analytics chart wrappers and local chart adapters should change. Backend APIs, analytics transforms, page orchestration, and shared UI models must remain stable.

External analytics engine rule:

If rAInty later replaces analytic-service with a market analytics engine, the integration must still normalize provider output into rAInty canonical analytics models before it reaches UserInterface pages. Market-engine query languages, chart specifications, or vendor-specific payloads must not become UserInterface contracts.

11. User settings for analytics​

Analytics and health indicators depend on user-configurable thresholds.

These settings must be stored in user_settings and exposed through data-service.

Candidate keys:

  • reconciliation.expected_expense_default_tolerance
  • reconciliation.expected_expense_fallback_window_days
  • reconciliation.utility_bill_default_tolerance
  • reconciliation.utility_bill_window_days
  • analytics.contract_payment_grace_days
  • analytics.expected_expense_grace_days

Guideline:

  • keep user-wide defaults first
  • add per-property or per-contract overrides only if clearly needed later

12. UI reuse strategy​

12.1 Shared UI building blocks​

Analytics presentation should reuse shared UI components.

Suggested atoms:

  • trend value
  • delta badge
  • health indicator
  • period label

Suggested molecules:

  • KPI card
  • breakdown row
  • health status card
  • dimension filter bar

Suggested organisms:

  • analytics KPI grid
  • expense breakdown section
  • contract health list
  • overdue items section
  • cashflow chart section

12.2 Page usage​

The same base datasets and transforms must feed:

  • Overview
  • Property detail
  • Bank account detail
  • Income expense report
  • future analytics sections in supplier or contract detail if needed

13. MVP implementation order​

Recommended execution order:

Phase A: foundations​

  1. define final analytics inclusion/exclusion rules
  2. define canonical fact schema
  3. define dedicated analytics schema and initial tables
  4. define refresh events and delta rules

Phase B: backend MVP​

  1. implement fact generation
  2. implement monthly aggregates
  3. implement contract health
  4. implement overview analytics API
  5. implement property summary API
  6. implement report API

Phase C: frontend MVP​

  1. create frontend analytics API file
  2. create transform engine foundation
  3. create analytics hooks
  4. create shared KPI / breakdown components
  5. wire Overview
  6. wire Property detail
  7. wire Income expense report

Phase D: optimization​

  1. add Redis cache for common analytics reads
  2. add scheduled partial rebuild
  3. add full rebuild admin path

14. Risks and design warnings​

14.1 What to avoid​

  • rebuilding all analytics after every change
  • page-specific duplicated aggregation logic
  • exposing raw operational tables for frontend analytics
  • putting analytics truth only in Redis
  • precomputing every possible dimension combination

14.2 Redis analytics cache​

Redis may be used only as a read-through cache after the first complete analytics query path is functional.

Initial cache key patterns:

  • analytics:overview:user:<id>:<scope-hash>
  • analytics:property:<id>:summary:user:<id>:<scope-hash>
  • analytics:contract-health:user:<id>:contract:<id>:<scope-hash>
  • analytics:bank-account:<id>:summary:user:<id>:<scope-hash>
  • analytics:report:user:<id>:<scope-hash>

The scope hash must include filters, dimensions, metrics, and authorizationContext.accessScope. This prevents data cached for one authorization scope from being reused for another user or sharing context.

Rules:

  • Redis is never source of truth
  • cache failures must not fail analytics reads
  • cached payloads must be canonical analytics payloads
  • cache TTL must be finite and configurable
  • invalidation can start with TTL-based expiry and later move to event-driven deletion when refresh/rebuild events are implemented

Configuration:

  • ANALYTICS_CACHE_ENABLED, default true
  • ANALYTICS_CACHE_TTL_SECONDS, default 120

14.3 Main consistency risks​

  • double counting due to both match and split paths
  • wrong competence month assignment
  • stale health indicators after manual override
  • cache invalidation gaps
  • partial resets leaving derived analytics stale

These must be explicitly tested.

15. Validation and testing strategy​

The analytics layer must be validated against representative cases:

  • single full reconciliation
  • multi split reconciliation
  • manual expected expense payment
  • utility bill payment with tolerance
  • reconciliation reset
  • ignored transaction excluded from economic facts
  • unreconciled transaction excluded from economic facts
  • draft or pending expense excluded from economic facts
  • overdue item becoming paid
  • contract moving from green to yellow to red

Validation should compare:

  • source operational rows
  • generated fact rows
  • aggregate totals
  • page payload output

16. Parallelization opportunities​

The following workstreams can be parallelized once the canonical fact contract is frozen.

16.1 Backend schema and foundation​

Can run in parallel:

  • analytics schema DDL
  • base analytics repository / module structure
  • event contract definition for refresh triggers

Dependency:

  • requires agreement on canonical fact model

16.2 Backend read models​

Can run in parallel:

  • overview analytics endpoint
  • property summary endpoint
  • contract health endpoint
  • bank account summary endpoint

Dependency:

  • requires canonical fact generation to be defined

16.3 Frontend analytics foundation​

Can run in parallel:

  • analytics API file
  • analytics type definitions
  • dimension registry
  • transform primitives
  • output view-model contracts

Dependency:

  • requires API payload contract draft

16.4 Shared UI components​

Can run in parallel:

  • KPI cards
  • breakdown sections
  • health components
  • chart wrappers

Dependency:

  • requires standard output models such as SummaryMetric[] and BreakdownRow[]

16.5 Page integration​

Can run in parallel after the shared frontend foundation exists:

  • Overview analytics integration
  • Property detail analytics integration
  • Income expense report analytics integration
  • Bank account detail analytics integration

16.6 Cache and scheduled rebuild​

Can run in parallel after core backend analytics APIs are stable:

  • Redis cache layer
  • scheduler partial rebuild job
  • admin rebuild tools
  1. freeze canonical fact contract
  2. split backend schema/facts from frontend transform foundation
  3. build shared UI components in parallel with backend read models
  4. integrate pages after both backend payloads and transform outputs are stable
  5. add cache and scheduled rebuild only after the first analytics flow is correct

17. Deployment note​

The current environment-sync-service schema migration path delegates execution to datahub.

At the moment, datahub/modules/schemaMigrationRunner.js accepts only a restricted additive DDL subset and rejects or does not support statements required by the initial analytics schema setup:

  • CREATE DATABASE
  • USE <schema>
  • CREATE USER
  • GRANT
  • schema-qualified CREATE TABLE statements such as rainty_analytics.analytics_economic_facts

For TEST deployment, the initial creation of rainty_analytics and its grants must therefore be executed by a database admin/deploy step, or the schema migration runner must be adapted first.

Required future adaptation:

  • allow a guarded CREATE DATABASE IF NOT EXISTS rainty_analytics
  • allow schema-qualified additive CREATE TABLE IF NOT EXISTS rainty_analytics.<table>
  • decide whether grants remain a DBA-only operation or are handled by an admin migration path
  • keep destructive operations forbidden
  • keep migration execution tracked in schema_migrations

Until this adaptation is complete, do not assume environment-sync-service can create the analytics schema in TEST.

18. TEST deployment checklist​

The first TEST deployment of analytics must verify infrastructure, internal authentication, and database state before functional validation.

Required checks:

  1. analytic-service is included in TEST deployment:
    • docker-compose.test.yml defines the analytic-service service.
    • COMPOSE_PROFILES includes analytic-service.
    • ANALYTIC_SERVICE_VERSION is available as a GitHub environment variable.
  2. Internal JWT keys are configured in the TEST GitHub environment:
    • INTERNAL_JWT_PRIVATE_KEY is available as a secret for data-service.
    • INTERNAL_JWT_PUBLIC_KEY is available as a secret for analytic-service.
    • both keys belong to the same key pair.
  3. data-service calls analytic-service internally through ANALYTIC_SERVICE_URL=http://analytic-service:3013.
  4. The rainty_analytics schema exists before analytics refresh/rebuild is executed.
  5. Until environment-sync-service and datahub support guarded cross-schema migrations, run the initial schema setup as a DBA/deploy-admin step:
CREATE DATABASE IF NOT EXISTS `rainty_analytics`;
GRANT ALL PRIVILEGES ON `rainty_analytics`.* TO 'rainty_user'@'%';
FLUSH PRIVILEGES;
  1. Apply db/migrations/001_analytic_service_schema.sql and later analytics migrations manually or through the adapted migration path.
  2. Verify the schema and derived tables in TEST:
SHOW DATABASES LIKE 'rainty_analytics';
SHOW TABLES FROM `rainty_analytics`;
  1. Verify service health and logs:
docker compose --env-file .env -f docker-compose.test.yml -p test ps analytic-service
docker logs --tail=100 test-analytic-service-1
  1. Verify the public analytics flow only through data-service with an authenticated user session. UserInterface must never call analytic-service directly.