Files
cal.diy/packages/features/calendar-subscription/README.md
T
e6b2116a2b feat: Calendar Cache and Sync (#23876)
* feat: calendar cache and sync - wip

* Add env.example

* refactor on CalendarCacheEventService

* remove test console.log

* Fix type checks errors

* chore: remove pt comment

* add route.ts

* chore: fix tests

* Improve cache impl

* chore: update recurring event id

* chore: small improvements

* calendar cache improvements

* Fix remove dynamic imports

* Add cleanup stale cache

* Fix tests

* add event update

* type fixes

* feat: add comprehensive tests for new calendar subscription API routes

- Add tests for /api/cron/calendar-subscriptions-cleanup route (9 tests)
- Add tests for /api/cron/calendar-subscriptions route (10 tests)
- Add tests for /api/webhooks/calendar-subscription/[provider] route (11 tests)
- Add missing feature flags for calendar-subscription-cache and calendar-subscription-sync
- All 30 tests pass with comprehensive coverage of authentication, feature flags, error handling, and service instantiation

Tests cover:
- Authentication scenarios (API key validation, Bearer tokens, query parameters)
- Feature flag combinations (cache/sync enabled/disabled states)
- Success and error handling (including non-Error exceptions)
- Service instantiation with proper dependency injection
- Provider validation for webhook endpoints

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* feat: add comprehensive tests for calendar subscription services, repositories, and adapters

- Add unit tests for CalendarSubscriptionService with subscription, webhook, and event processing
- Add unit tests for CalendarCacheEventService with cache operations and cleanup
- Add unit tests for CalendarSyncService with Cal.com event filtering and booking operations
- Add unit tests for CalendarCacheEventRepository with CRUD operations
- Add unit tests for SelectedCalendarRepository with calendar selection management
- Add unit tests for GoogleCalendarSubscriptionAdapter with subscription and event fetching
- Add unit tests for Office365CalendarSubscriptionAdapter with placeholder implementation
- Add unit tests for AdaptersFactory with provider management and adapter creation
- Fix lint issues by removing explicit 'any' type casting and unused variables
- All tests follow Cal.com conventions using Vitest framework with proper mocking

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* fix: improve calendar-subscriptions-cleanup test performance by adding missing mocks

- Add comprehensive mocks for defaultResponderForAppDir, logger, performance monitoring, and Sentry
- Fix slow test execution (933ms -> <100ms) caused by missing dependency mocks
- Ensure consistent test performance across different environments

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* Fix tests

* Fix tests

* type fix

* Fix coderabbit comments

* Fix types

* Fix test

* Update apps/web/app/api/cron/calendar-subscriptions/route.ts

Co-authored-by: Alex van Andel <me@alexvanandel.com>

* Fixes by first review

* feat: add database migrations for calendar cache and sync fields

- Add CalendarCacheEventStatus enum with confirmed, tentative, cancelled values
- Add new fields to SelectedCalendar: channelId, channelKind, channelResourceId, channelResourceUri, channelExpiration, syncSubscribedAt, syncToken, syncedAt, syncErrorAt, syncErrorCount
- Create CalendarCacheEvent table with foreign key to SelectedCalendar
- Add necessary indexes and constraints for performance and data integrity

Fixes database schema issues causing e2e test failures with 'column does not exist' errors.

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* only google-calendar for now

* docs: add Calendar Cache and Sync feature documentation

- Add comprehensive feature overview and motivation
- Document feature flags with SQL examples
- Include SQL examples for enabling features for users and teams
- Reference technical documentation files

Addresses PR #23876 documentation requirements

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* docs: update calendar subscription README with comprehensive documentation

- Undo incorrect changes to main README.md
- Update packages/features/calendar-subscription/README.md with:
  - Feature overview and motivation
  - Environment variables section
  - Complete feature flags documentation with SQL examples
  - SQL examples for enabling features for users and teams
  - Detailed architecture documentation

Addresses PR #23876 documentation requirements

Co-Authored-By: Volnei Munhoz <volnei.munhoz@gmail.com>

* fix docs

* Fix test to available calendars

* Fix test to available calendars

* add migration and sync boilerplate

* fix typo

* remove double log

* sync boilerplate

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Alex van Andel <me@alexvanandel.com>
Co-authored-by: Keith Williams <keithwillcode@gmail.com>
2025-09-29 14:26:14 +00:00

116 lines
5.4 KiB
Markdown

# Calendar Cache and Sync
The **Calendar Cache and Sync** feature provides efficient calendar synchronization with intelligent caching to reduce API calls and ensure real-time updates across your Cal.com instance.
## Feature Overview
This feature introduces two complementary capabilities:
- **Calendar Cache**: Stores calendar availability data locally to reduce external API calls and improve performance
- **Calendar Sync**: Uses webhooks to automatically listen for calendar updates and apply changes in real-time
**Key Benefits:**
- **Efficiency**: Reduces API calls with optimized caching strategies
- **Reliability**: Guarantees updates through webhook event delivery
- **Real-Time Sync**: Ensures calendars are always up-to-date with minimal latency
- **Scalability**: Supports multiple calendars and handles high-volume updates seamlessly
**Motivation:**
By subscribing to calendars via webhooks and implementing intelligent caching, you gain a smarter, faster, and more resource-friendly way to keep your data in sync. This eliminates the need for constant polling and reduces the load on external calendar APIs while ensuring data consistency.
## Environment Variables
- **GOOGLE_WEBHOOK_URL**: Optional, only used for local tests. Default points to application url.
- **GOOGLE_WEBHOOK_TOKEN**: Required, token to validate Google Webhook incoming.
- **MICROSOFT_WEBHOOK_URL**: Optional, only used for local tests. Default points to application url.
- **MICROSOFT_WEBHOOK_TOKEN**: Required, token to validate Microsoft Webhook incoming.
## Feature Flags
This feature is controlled by three feature flags that can be enabled independently:
### 1. calendar-subscription-cache
Enables calendar cache recording and usage through calendars. This flag should be managed individually by teams.
```sql
INSERT INTO "Feature" ("slug", "enabled", "description", "type", "stale", "lastUsedAt", "createdAt", "updatedAt", "updatedBy")
VALUES ('calendar-subscription-cache', false, 'Allow calendar cache to be recorded and used through calendars.', 'OPERATIONAL', false, NULL, NOW(), NOW(), NULL)
ON CONFLICT (slug) DO NOTHING;
```
### 2. calendar-subscription-sync
Enables calendar sync globally for all users regardless of team or organization.
```sql
INSERT INTO "Feature" ("slug", "enabled", "description", "type", "stale", "lastUsedAt", "createdAt", "updatedAt", "updatedBy")
VALUES ('calendar-subscription-sync', false, 'Enable calendar sync for all calendars globally.', 'OPERATIONAL', false, NULL, NOW(), NOW(), NULL)
ON CONFLICT (slug) DO NOTHING;
```
## Enabling Features for Specific Users
To enable calendar cache features for specific users, add entries to the `UserFeatures` table:
```sql
-- Enable calendar-subscription-cache for user ID 123
INSERT INTO "UserFeatures" ("userId", "featureId", "assignedAt", "assignedBy", "updatedAt")
VALUES (123, 'calendar-subscription-cache', NOW(), 'admin', NOW())
ON CONFLICT ("userId", "featureId") DO NOTHING;
-- Enable calendar-subscription-sync for user ID 123
INSERT INTO "UserFeatures" ("userId", "featureId", "assignedAt", "assignedBy", "updatedAt")
VALUES (123, 'calendar-subscription-sync', NOW(), 'admin', NOW())
ON CONFLICT ("userId", "featureId") DO NOTHING;
```
## Enabling Features for Specific Teams
To enable calendar cache features for specific teams, add entries to the `TeamFeatures` table:
```sql
-- Enable calendar-subscription-cache for team ID 456
INSERT INTO "TeamFeatures" ("teamId", "featureId", "assignedAt", "assignedBy", "updatedAt")
VALUES (456, 'calendar-subscription-cache', NOW(), 'admin', NOW())
ON CONFLICT ("teamId", "featureId") DO NOTHING;
-- Enable calendar-subscription-sync for team ID 456
INSERT INTO "TeamFeatures" ("teamId", "featureId", "assignedAt", "assignedBy", "updatedAt")
VALUES (456, 'calendar-subscription-sync', NOW(), 'admin', NOW())
ON CONFLICT ("teamId", "featureId") DO NOTHING;
```
## Architecture
The calendar cache and sync system consists of several key components:
### Database Schema
- **CalendarCacheEvent Table**: Stores cached calendar events with status tracking
- **SelectedCalendar Extensions**: Additional fields for sync state and webhook management including:
- `channelId`: Webhook channel identifier
- `channelResourceId`: Resource ID for webhook subscriptions
- `channelResourceUri`: URI for webhook notifications
- `channelKind`: Type of webhook channel
- `channelExpiration`: Webhook subscription expiration time
- `syncToken`: Token for incremental sync
- `syncedAt`: Last successful sync timestamp
- `syncErrorAt`: Last sync error timestamp
- `syncErrorCount`: Number of consecutive sync errors
- `syncSubscribedAt`: Webhook subscription timestamp
### Core Services
- **CalendarCacheEventRepository**: Manages cached event storage and retrieval
- **CalendarSubscriptionService**: Orchestrates webhook subscriptions and event processing
- **Provider-specific Adapters**: Handle calendar-specific sync logic (Google, Office365)
### Background Processes
- **Cron Jobs**: Automated processes for cache cleanup and calendar watching
- **Webhook Handlers**: Real-time event processing for calendar updates
### Integration Points
- **Calendar Providers**: Google Calendar, Office365, and other supported integrations
- **Webhook Endpoints**: Receive real-time notifications from calendar providers
- **Cache Layer**: Optimized storage for frequently accessed calendar data
For detailed technical implementation, see:
- Database migrations in `packages/prisma/migrations/`