diff --git a/MIGRATION_PLAN_PHP_TO_JAVA.md b/MIGRATION_PLAN_PHP_TO_JAVA.md new file mode 100644 index 0000000..b9fd98b --- /dev/null +++ b/MIGRATION_PLAN_PHP_TO_JAVA.md @@ -0,0 +1,722 @@ +# Migration Plan: PHP to Java + +**Status**: Planning Phase +**Created**: 2026-02-06 +**Target Framework**: Spring Boot 3.x +**Build Tool**: Maven (recommended for enterprise Java applications) + +## Executive Summary + +This document outlines a comprehensive plan to migrate the Offpost email thread management system from PHP to Java. The application currently consists of 181 PHP files with a complex architecture involving email processing, IMAP integration, AI services, and multiple database backends. + +## Current Architecture Overview + +### Technology Stack (PHP) +- **Language**: PHP 8.x +- **Web Server**: Apache (via Docker) +- **Database**: PostgreSQL 15 (main app) + MySQL 5.6 (Roundcube) +- **Email**: IMAP (GreenMail in dev), SendGrid (production) +- **AI**: OpenAI API integration +- **Authentication**: Auth0 (production), custom service (dev) +- **Testing**: PHPUnit 10.5 +- **Dependencies**: PHPMailer, Laminas Mail, PDO + +### Core Components +1. **Organizer** - Main PHP application (181 files) + - 57 class files in `organizer/src/class/` + - API endpoints in `organizer/src/api/` + - Web pages for UI + - Scheduled tasks in `organizer/src/system-pages/` + - 30 SQL migrations + +2. **Roundcube** - Webmail client (Docker container) +3. **IMAP Server** - Email storage (GreenMail/production server) +4. **PostgreSQL** - Main application database +5. **MySQL** - Roundcube database +6. **Auth Service** - Development authentication + +## Target Architecture (Java) + +### Technology Stack (Java) +- **Language**: Java 21 LTS +- **Framework**: Spring Boot 3.2+ +- **Build Tool**: Maven 3.9+ +- **Database**: PostgreSQL 15 (main app) + MySQL 5.6 (Roundcube - unchanged) +- **ORM**: Spring Data JPA with Hibernate +- **Migration**: Flyway or Liquibase +- **Email**: JavaMail API for IMAP, SendGrid Java SDK +- **AI**: OpenAI Java SDK or HTTP client +- **Authentication**: Spring Security with Auth0 integration +- **Testing**: JUnit 5, Mockito, TestContainers, GreenMail +- **Scheduling**: Spring @Scheduled annotations +- **API**: Spring REST with OpenAPI/Swagger documentation + +### Recommended Project Structure +``` +offpost-java/ +├── pom.xml +├── src/ +│ ├── main/ +│ │ ├── java/ +│ │ │ └── no/offpost/ +│ │ │ ├── domain/ # Domain entities +│ │ │ │ ├── Thread.java +│ │ │ │ ├── ThreadEmail.java +│ │ │ │ ├── Entity.java +│ │ │ │ └── enums/ +│ │ │ ├── repository/ # JPA repositories +│ │ │ ├── service/ # Business logic +│ │ │ │ ├── imap/ +│ │ │ │ ├── email/ +│ │ │ │ ├── ai/ +│ │ │ │ └── extraction/ +│ │ │ ├── controller/ # REST API +│ │ │ ├── scheduler/ # Scheduled tasks +│ │ │ ├── config/ # Configuration +│ │ │ └── Application.java +│ │ └── resources/ +│ │ ├── application.yml +│ │ ├── db/migration/ # Flyway migrations +│ │ └── templates/ # Thymeleaf (optional) +│ └── test/ +│ └── java/ +│ └── no/offpost/ +│ ├── unit/ +│ ├── integration/ +│ └── e2e/ +├── Dockerfile +└── README.md +``` + +## Migration Phases + +### Phase 1: Foundation and Setup (Week 1-2) + +**Objective**: Establish Java project infrastructure + +**Tasks**: +1. Create Maven project with Spring Boot starter +2. Set up project structure (package hierarchy) +3. Configure `pom.xml` with dependencies: + - Spring Boot Starter Web + - Spring Boot Starter Data JPA + - PostgreSQL JDBC driver + - MySQL JDBC driver (for Roundcube) + - JavaMail API + - Spring Boot Starter Security + - Spring Boot Starter Test + - TestContainers + - Lombok (for reducing boilerplate) +4. Create `application.yml` with environment-specific profiles +5. Set up Docker Compose for Java development environment +6. Configure logging (SLF4J with Logback) +7. Implement health check endpoints + +**Deliverables**: +- Running Spring Boot application +- Docker Compose configuration +- Basic project structure +- CI/CD pipeline configuration + +### Phase 2: Core Domain Model (Week 3-4) + +**Objective**: Port domain entities and business objects + +**Tasks**: +1. Create `Thread` entity with JPA annotations + - Status enums: `ThreadSendingStatus` (STAGING, READY_FOR_SENDING, SENDING, SENT) + - Law basis enums: `RequestLawBasis` (OFFENTLEGLOVA, OTHER) + - Follow-up plan enums: `RequestFollowUpPlan` (SPEEDY, SLOW) + - All properties from PHP version + - Proper JPA relationships + +2. Create `ThreadEmail` entity with attachments relationship +3. Create `ThreadEmailAttachment` entity +4. Create `Entity` class (public organization) +5. Create `Identity` class +6. Create `ImapFolderStatus` class +7. Create `ThreadHistory` class +8. Create `ImapFolderLog` class +9. Implement proper equals/hashCode/toString +10. Add validation annotations (JSR-303) + +**PHP Files to Port**: +- `class/Thread.php` +- `class/ThreadEmail.php` +- `class/ThreadEmailAttachment.php` +- `class/Entity.php` +- `class/Identity.php` +- `class/ImapFolderStatus.php` +- `class/ThreadHistory.php` +- `class/ImapFolderLog.php` + +**Deliverables**: +- Complete domain model in Java +- Unit tests for domain logic +- JSON serialization/deserialization working + +### Phase 3: Database Layer (Week 5-6) + +**Objective**: Implement database access and migration + +**Tasks**: +1. Port 30 SQL migrations to Flyway format (`V1__*.sql` to `V30__*.sql`) +2. Create JPA repositories: + - `ThreadRepository extends JpaRepository` + - `ThreadEmailRepository` + - `EntityRepository` + - `IdentityRepository` + - `ImapFolderStatusRepository` + - `ImapFolderLogRepository` +3. Port `ThreadDatabaseOperations` to service layer +4. Port `ThreadStatusRepository` with custom queries +5. Implement `ThreadStorageManager` functionality +6. Configure transaction management +7. Set up connection pooling (HikariCP) + +**PHP Files to Port**: +- `class/Database.php` → Spring Data JPA configuration +- `class/ThreadDatabaseOperations.php` → `ThreadService.java` +- `class/ThreadStatusRepository.php` → Custom repository methods +- `class/ThreadStorageManager.php` → Service layer +- `migrations/sql/*.sql` → `db/migration/*.sql` + +**Deliverables**: +- All migrations ported and tested +- Repository layer complete with tests +- Database integration tests using TestContainers + +### Phase 4: IMAP Integration (Week 7-9) + +**Objective**: Implement email handling with IMAP + +**Tasks**: +1. Create `ImapWrapper` using JavaMail API + - Connection management with retry logic + - Exponential backoff for transient failures + - Folder operations (list, create, rename, delete) + - Message operations (fetch, move, delete) + +2. Port `ImapConnection` class + - Connection pooling + - SSL/TLS configuration + - Credential management + +3. Port `ImapFolderManager` + - Folder hierarchy management + - Thread-specific folder operations + +4. Port `ImapEmail` and `ImapEmailProcessor` + - Email parsing from IMAP + - Header extraction + - Body parsing (text/HTML) + +5. Port `ImapAttachmentHandler` + - Attachment extraction + - Binary data handling + - Content type detection + +6. Port `ThreadEmailMover` + - Move emails between folders + - Error handling and recovery + +**PHP Files to Port**: +- `class/Imap/ImapWrapper.php` +- `class/Imap/ImapConnection.php` +- `class/Imap/ImapFolderManager.php` +- `class/Imap/ImapEmail.php` +- `class/Imap/ImapEmailProcessor.php` +- `class/Imap/ImapAttachmentHandler.php` +- `class/ThreadEmailMover.php` +- `class/ThreadFolderManager.php` + +**Deliverables**: +- Complete IMAP integration +- Integration tests with GreenMail +- Retry logic verified under failure conditions + +### Phase 5: Email Processing Services (Week 10-12) + +**Objective**: Implement scheduled email processing tasks + +**Tasks**: +1. Port `ThreadScheduledEmailReceiver` + - Spring @Scheduled annotation + - Poll IMAP for new emails + - Process incoming messages + - Error handling and logging + +2. Port `ThreadScheduledEmailSender` + - Process sending queue + - SendGrid integration + - Status updates + - Failure handling + +3. Port `ThreadScheduledFollowUpSender` + - Automatic follow-up logic + - Scheduling based on follow-up plan + - Follow-up email generation + +4. Port `ThreadEmailDatabaseSaver` + - Parse EML files + - Extract headers, body, attachments + - Save to database + - Handle multipart messages + +5. Port `ThreadEmailSending` and `ThreadEmailService` + - Email composition + - Template handling + - Sending via SendGrid + - Copy to IMAP sent folder + +6. Implement SendGrid integration + - Configure SendGrid Java SDK + - Event webhooks for delivery status + - Error handling + +**PHP Files to Port**: +- `class/ThreadScheduledEmailReceiver.php` +- `class/ThreadScheduledEmailSender.php` +- `class/ThreadScheduledFollowUpSender.php` +- `class/ThreadEmailDatabaseSaver.php` +- `class/ThreadEmailSending.php` +- `class/ThreadEmailService.php` +- `system-pages/scheduled-email-sending.php` +- `system-pages/scheduled-email-receiver.php` +- `system-pages/scheduled-imap-handling.php` +- `system-pages/scheduled-thread-follow-up.php` + +**Deliverables**: +- Scheduled tasks running in Spring +- Email sending/receiving working +- Follow-up automation functional +- E2E tests for email processing + +### Phase 6: AI Integration (Week 13-14) + +**Objective**: Integrate OpenAI for email processing + +**Tasks**: +1. Port OpenAI integration classes + - HTTP client for OpenAI API + - Request/response models + - Error handling and retry logic + - Token usage tracking + +2. Port `SuggestedReplyGenerator` + - Generate reply suggestions using GPT + - Context from email thread + - Template customization + +3. Port `ThreadEmailClassifier` + - Classify email types + - Extract key information + - Sentiment analysis + +4. Port extraction services + - PDF text extraction (using Apache PDFBox) + - Image text extraction (OCR if needed) + - HTML to text conversion + - Content summarization + +5. Configure OpenAI client + - API key management + - Rate limiting + - Cost tracking + +**PHP Files to Port**: +- `class/Ai/*.php` (all AI-related classes) +- `class/Extraction/*.php` (all extraction services) +- `class/SuggestedReplyGenerator.php` +- `class/ThreadEmailClassifier.php` +- `system-pages/scheduled-email-extraction.php` +- `api/thread_email_extraction.php` + +**Deliverables**: +- OpenAI integration working +- Extraction services functional +- Unit tests with mocked API responses +- Integration tests with test API key + +### Phase 7: REST API Layer (Week 15-16) + +**Objective**: Create REST API for frontend/external access + +**Tasks**: +1. Create Spring REST controllers: + - `ThreadController` - CRUD operations for threads + - `EmailController` - Email operations + - `EntityController` - Public entity management + - `ExtractionController` - AI extraction endpoints + - `AdminController` - Administrative operations + +2. Port authentication logic + - Spring Security configuration + - Auth0 integration (production) + - JWT token validation + - Role-based access control + +3. Implement `ThreadAuthorization` + - Check user permissions + - Thread access control + - Public thread handling + +4. Port API endpoints from `api/` directory + - Thread creation/update/delete + - Email operations + - Extraction triggers + - Bulk operations + +5. Implement `AdminNotificationService` + - Alert on critical errors + - Email notifications + - Logging integration + +6. Add OpenAPI/Swagger documentation + - API documentation UI + - Request/response schemas + - Example payloads + +**PHP Files to Port**: +- `api/*.php` (all API endpoints) +- `class/ThreadAuthorization.php` +- `class/AdminNotificationService.php` +- `auth.php` +- `callback.php` + +**Deliverables**: +- Complete REST API +- API documentation (Swagger UI) +- Authentication working +- Integration tests for all endpoints + +### Phase 8: Web Frontend (Week 17-20) + +**Objective**: Provide web UI for the application + +**Options to Consider**: + +**Option A: Keep PHP Pages (Temporary)** +- Minimal changes to existing PHP pages +- API calls to Java backend +- Gradual migration path +- Good for quick transition + +**Option B: Migrate to Thymeleaf** +- Server-side rendering with Spring +- Template-based approach +- Similar to PHP pages +- Less frontend complexity + +**Option C: Separate SPA (React/Vue/Angular)** +- Modern frontend framework +- Better user experience +- Complete separation of concerns +- More development effort + +**Recommended Approach**: Option B (Thymeleaf) or Option C (SPA) + +**Tasks for Option B (Thymeleaf)**: +1. Set up Thymeleaf templates +2. Port page logic to Spring MVC controllers: + - Index page (thread listing) + - Thread view page + - Thread creation page + - Thread reply page + - Entity management pages + - Extraction overview pages + - Bulk action pages +3. Port forms and validation +4. Handle file uploads +5. Implement session management +6. Port Roundcube integration logic + +**PHP Files to Port**: +- `index.php` +- `view-thread.php` +- `start-thread.php` +- `thread-reply.php` +- `entities.php` +- `thread-bulk-actions.php` +- `update-identities.php` +- `update-imap.php` +- `recent-activity.php` +- `grant-thread-access.php` +- And all other UI pages + +**Deliverables**: +- Functional web UI +- Feature parity with PHP version +- E2E tests for web pages +- User documentation + +### Phase 9: Testing Infrastructure (Week 21-22) + +**Objective**: Achieve comprehensive test coverage + +**Tasks**: +1. Set up JUnit 5 and Mockito +2. Port unit tests: + - Domain model tests + - Service layer tests + - Repository tests + - IMAP integration tests + - Email processing tests + - AI integration tests +3. Port integration tests with TestContainers: + - PostgreSQL container + - GreenMail container + - Full application context tests +4. Port E2E tests: + - Web page tests (Selenium/Playwright) + - API endpoint tests + - Email flow tests +5. Configure test coverage reporting (JaCoCo) +6. Set up continuous testing in CI/CD + +**PHP Test Files to Port** (57+ test files): +- `tests/*.php` (all unit tests) +- `e2e-tests/pages/*.php` (all E2E tests) + +**Test Requirements to Maintain**: +- Tests must be deterministic (no random values, no time-based data) +- Tests must fail instead of skip (no `@Disabled` without good reason) +- Use exact assertions for deterministic outputs +- Follow Arrange-Act-Assert pattern +- Each test should be independent + +**Deliverables**: +- Complete test suite (unit + integration + E2E) +- Test coverage > 80% +- All tests passing +- CI/CD integration + +### Phase 10: Deployment and Migration (Week 23-24) + +**Objective**: Deploy Java application and migrate data + +**Tasks**: +1. Create production Dockerfile: + - Multi-stage build for efficiency + - JRE base image (Eclipse Temurin 21) + - Non-root user + - Health checks + +2. Update Docker Compose files: + - `docker-compose.dev.yaml` - Java development environment + - `docker-compose.prod.yaml` - Production configuration + +3. Create Kubernetes manifests (if applicable): + - Deployment + - Service + - ConfigMap + - Secret + - Ingress + +4. Migration strategy: + - Database migration scripts (if schema changes) + - Data validation scripts + - Rollback procedures + +5. Deployment approach: + - **Blue-Green Deployment**: Run both versions, switch traffic + - **Canary Deployment**: Gradually shift traffic to Java + - **Feature Flags**: Control which version handles requests + +6. Monitoring and observability: + - Application metrics (Micrometer) + - Logs (structured logging) + - Health endpoints + - Performance monitoring + +7. Production cutover plan: + - Backup procedures + - Validation checkpoints + - Rollback triggers + - Communication plan + +**Deliverables**: +- Production-ready Docker images +- Deployment automation +- Migration scripts +- Monitoring setup +- Cutover runbook + +### Phase 11: Documentation (Week 25-26) + +**Objective**: Comprehensive documentation for Java version + +**Tasks**: +1. Update `README.md`: + - Java/Maven setup instructions + - Build commands + - Development workflow + - Deployment procedures + +2. Update `CLAUDE.md` (or create `CLAUDE_JAVA.md`): + - Java-specific conventions + - Package structure + - Testing guidelines + - Common tasks + +3. Create Java-specific documentation: + - Architecture Decision Records (ADRs) + - API documentation (Swagger/OpenAPI) + - Database schema documentation + - Email flow diagrams + - Troubleshooting guide + +4. Developer onboarding guide: + - IDE setup (IntelliJ IDEA / Eclipse) + - Code style configuration + - Git workflow + - Testing strategy + +5. Operations runbook: + - Deployment procedures + - Monitoring and alerting + - Backup and recovery + - Common issues and solutions + +**Deliverables**: +- Complete documentation set +- Updated repository README +- Developer onboarding guide +- Operations runbook + +## Key Technical Decisions + +### 1. Build Tool: Maven vs Gradle + +**Recommendation: Maven** +- More widely used in enterprise Java +- Better IDE support +- Simpler for standard Spring Boot projects +- Larger ecosystem + +### 2. Framework: Spring Boot + +**Rationale**: +- Industry standard for Java web applications +- Comprehensive ecosystem (Data, Security, Cloud) +- Excellent documentation and community +- Built-in support for scheduled tasks, REST APIs, JPA +- Easy integration with Auth0, SendGrid, OpenAI + +### 3. Migration Tool: Flyway vs Liquibase + +**Recommendation: Flyway** +- Simpler SQL-based approach +- Easier migration from existing SQL scripts +- Good Spring Boot integration +- Sufficient for this use case + +### 4. Frontend Approach + +**Recommendation: Thymeleaf for MVP, consider SPA later** +- Thymeleaf provides quickest path to feature parity +- Server-side rendering reduces complexity +- Can transition to SPA incrementally +- Consider React/Vue for future enhancement + +### 5. Scheduling: Spring @Scheduled + +**Rationale**: +- Built into Spring Framework +- Simple annotation-based configuration +- Sufficient for current cron-like tasks +- Can scale to Quartz if needed + +### 6. Email Library: JavaMail API + +**Rationale**: +- Standard Java email API +- Comprehensive IMAP support +- Good Spring integration +- Well-documented + +## Risk Analysis and Mitigation + +### Technical Risks + +**Risk 1: IMAP Behavior Differences** +- **Impact**: Email processing might behave differently +- **Mitigation**: Comprehensive integration tests with GreenMail; parallel run to compare behavior + +**Risk 2: Performance Degradation** +- **Impact**: Java application might be slower or use more memory +- **Mitigation**: Performance testing early; profiling and optimization; proper JVM tuning + +**Risk 3: Data Migration Issues** +- **Impact**: Data loss or corruption during migration +- **Mitigation**: Thorough testing of migration scripts; backup procedures; ability to rollback + +**Risk 4: Third-Party Integration Changes** +- **Impact**: Different behavior with Auth0, SendGrid, OpenAI +- **Mitigation**: Integration tests; sandbox testing; gradual rollout + +**Risk 5: Missing PHP Functionality** +- **Impact**: Some PHP-specific features might be hard to replicate +- **Mitigation**: Early identification of PHP-specific code; find Java alternatives; document workarounds + +### Project Risks + +**Risk 6: Scope Creep** +- **Impact**: Migration takes longer than planned +- **Mitigation**: Strict feature parity focus; defer enhancements; clear phase boundaries + +**Risk 7: Knowledge Gap** +- **Impact**: Team unfamiliar with Java/Spring ecosystem +- **Mitigation**: Training and documentation; pair programming; code reviews + +**Risk 8: Testing Coverage Gap** +- **Impact**: Bugs in production due to incomplete testing +- **Mitigation**: Port tests early; maintain coverage metrics; E2E testing + +## Success Criteria + +1. **Functional Parity**: All features from PHP version working in Java +2. **Test Coverage**: >80% code coverage with passing tests +3. **Performance**: Response times comparable to PHP version +4. **Stability**: No critical bugs in first month of production +5. **Documentation**: Complete and accurate documentation +6. **Deployment**: Successful production deployment with rollback capability + +## Timeline Estimate + +**Total Duration**: 26 weeks (6.5 months) + +- Phase 1-2: Foundation and Domain (4 weeks) +- Phase 3-4: Database and IMAP (5 weeks) +- Phase 5-6: Email Processing and AI (5 weeks) +- Phase 7-8: API and Frontend (6 weeks) +- Phase 9-11: Testing, Deployment, Documentation (6 weeks) + +**Note**: Timeline is aggressive and assumes: +- Full-time dedicated resources +- Team familiar with Java/Spring Boot +- Minimal scope changes +- Parallel work where possible + +## Resource Requirements + +- **Java Developers**: 2-3 full-time +- **DevOps Engineer**: 0.5 FTE (part-time) +- **QA Engineer**: 1 full-time +- **Technical Writer**: 0.25 FTE (part-time) + +## Next Steps + +1. **Review and Approve Plan**: Stakeholder review of this plan +2. **Resource Allocation**: Assign team members +3. **Environment Setup**: Prepare development infrastructure +4. **Phase 1 Kickoff**: Begin foundation work +5. **Weekly Reviews**: Track progress against plan + +## References + +- [Spring Boot Documentation](https://spring.io/projects/spring-boot) +- [JavaMail API Guide](https://javaee.github.io/javamail/) +- [Flyway Documentation](https://flywaydb.org/documentation/) +- [TestContainers](https://www.testcontainers.org/) +- [Auth0 Spring Security Integration](https://auth0.com/docs/quickstart/webapp/java-spring-boot) diff --git a/docs/migration/GETTING_STARTED_JAVA.md b/docs/migration/GETTING_STARTED_JAVA.md new file mode 100644 index 0000000..ecae2e3 --- /dev/null +++ b/docs/migration/GETTING_STARTED_JAVA.md @@ -0,0 +1,624 @@ +# Getting Started with Java Migration + +This guide provides immediate next steps for starting the PHP to Java migration. + +## Quick Start + +### Prerequisites + +- Java 21 LTS ([Download](https://adoptium.net/)) +- Maven 3.9+ ([Download](https://maven.apache.org/download.cgi)) +- Docker and Docker Compose +- IDE: IntelliJ IDEA (recommended) or Eclipse + +### Step 1: Create Spring Boot Project + +```bash +# Using Spring Initializr CLI (or visit https://start.spring.io/) +curl https://start.spring.io/starter.zip \ + -d dependencies=web,data-jpa,postgresql,security,mail,validation,actuator \ + -d groupId=no.offpost \ + -d artifactId=offpost \ + -d name=Offpost \ + -d description="Email thread management system for public entities" \ + -d packageName=no.offpost \ + -d javaVersion=21 \ + -d type=maven-project \ + -o offpost-java.zip + +unzip offpost-java.zip -d offpost-java +cd offpost-java +``` + +### Step 2: Add Additional Dependencies + +Edit `pom.xml` to add: + +```xml + + + + org.springframework.boot + spring-boot-starter-web + + + org.springframework.boot + spring-boot-starter-data-jpa + + + org.postgresql + postgresql + runtime + + + + + com.sun.mail + jakarta.mail + 2.0.1 + + + com.sendgrid + sendgrid-java + 4.10.2 + + + org.flywaydb + flyway-core + + + org.projectlombok + lombok + true + + + com.auth0 + auth0-spring-security-api + 1.5.3 + + + + + org.springframework.boot + spring-boot-starter-test + test + + + org.testcontainers + testcontainers + 1.19.3 + test + + + org.testcontainers + postgresql + 1.19.3 + test + + + com.icegreen + greenmail-junit5 + 2.0.1 + test + + +``` + +### Step 3: Configure Application Properties + +Create `src/main/resources/application.yml`: + +```yaml +spring: + application: + name: Offpost + datasource: + url: jdbc:postgresql://localhost:5432/offpost + username: offpost + password: ${DB_PASSWORD} + jpa: + hibernate: + ddl-auto: validate + show-sql: false + properties: + hibernate: + format_sql: true + flyway: + enabled: true + locations: classpath:db/migration + +# IMAP Configuration +imap: + server: ${IMAP_SERVER:localhost} + port: ${IMAP_PORT:993} + email: ${IMAP_EMAIL} + password: ${IMAP_PASSWORD} + ssl: true + +# SendGrid Configuration +sendgrid: + api-key: ${SENDGRID_API_KEY} + +# OpenAI Configuration +openai: + api-key: ${OPENAI_API_KEY} + +# Auth0 Configuration +auth0: + domain: ${AUTH0_DOMAIN} + client-id: ${AUTH0_CLIENT_ID} + client-secret: ${AUTH0_CLIENT_SECRET} + +# Management endpoints +management: + endpoints: + web: + exposure: + include: health,info,metrics +``` + +### Step 4: Create Basic Domain Model + +Start with the core `Thread` entity: + +```java +package no.offpost.domain; + +import jakarta.persistence.*; +import lombok.Data; +import java.time.Instant; +import java.util.ArrayList; +import java.util.List; +import java.util.UUID; + +/** + * Core entity representing an email thread with a public entity. + * + *

A Thread represents a conversation between the system and a public entity, + * with a unique profile (name and email) that is automatically generated for each thread. + * The thread manages email lifecycle through status transitions: STAGING → READY_FOR_SENDING → SENDING → SENT. + * + * @author Offpost Team + * @see ThreadEmail + * @see SendingStatus + */ +@Entity +@Table(name = "threads") +@Data +public class Thread { + + /** + * Unique identifier for the thread (UUID). + */ + @Id + private String id; + + /** + * ID of the public entity this thread communicates with. + */ + @Column(name = "entity_id") + private Integer entityId; + + /** + * Title or subject of the thread. + */ + private String title; + + /** + * Randomly generated name used as sender identity for this thread. + */ + @Column(name = "my_name") + private String myName; + + /** + * Unique email address generated for this thread. + */ + @Column(name = "my_email") + private String myEmail; + + /** + * Comma-separated labels for categorizing the thread. + */ + private String labels; + + /** + * Current sending status of the thread. + */ + @Column(name = "sending_status") + @Enumerated(EnumType.STRING) + private SendingStatus sendingStatus; + + /** + * Initial request text sent to the public entity. + */ + @Column(name = "initial_request", columnDefinition = "TEXT") + private String initialRequest; + + /** + * Whether the thread has been archived. + */ + private Boolean archived = false; + + /** + * Whether the thread is publicly accessible. + */ + @Column(name = "public") + private Boolean publicThread = false; + + /** + * Comment added when the email was sent. + */ + @Column(name = "sent_comment") + private String sentComment; + + /** + * Legal basis for the request (e.g., Norwegian Freedom of Information Act). + */ + @Column(name = "request_law_basis") + @Enumerated(EnumType.STRING) + private RequestLawBasis requestLawBasis; + + /** + * Follow-up plan determining how aggressively to follow up (speedy or slow). + */ + @Column(name = "request_follow_up_plan") + @Enumerated(EnumType.STRING) + private RequestFollowUpPlan requestFollowUpPlan; + + /** + * Timestamp when the thread was created. + */ + @Column(name = "created_at") + private Instant createdAt; + + /** + * Timestamp when the thread was last updated. + */ + @Column(name = "updated_at") + private Instant updatedAt; + + /** + * All emails associated with this thread. + */ + @OneToMany(mappedBy = "thread", cascade = CascadeType.ALL) + private List emails = new ArrayList<>(); + + /** + * JPA lifecycle callback executed before persisting a new thread. + * Initializes ID, timestamps, and default sending status. + */ + @PrePersist + protected void onCreate() { + if (id == null) { + id = UUID.randomUUID().toString(); + } + createdAt = Instant.now(); + updatedAt = Instant.now(); + if (sendingStatus == null) { + sendingStatus = SendingStatus.STAGING; + } + } + + /** + * JPA lifecycle callback executed before updating a thread. + * Updates the updatedAt timestamp. + */ + @PreUpdate + protected void onUpdate() { + updatedAt = Instant.now(); + } + + /** + * Email sending status indicating the thread's lifecycle stage. + */ + public enum SendingStatus { + /** Thread is being prepared, not ready to send */ + STAGING, + /** Thread is ready to be sent */ + READY_FOR_SENDING, + /** Thread is currently being sent */ + SENDING, + /** Thread has been successfully sent */ + SENT + } + + /** + * Legal basis for the information request. + */ + public enum RequestLawBasis { + /** Norwegian Freedom of Information Act (Offentleglova) */ + OFFENTLEGLOVA, + /** Other legal basis */ + OTHER + } + + /** + * Follow-up strategy for the thread. + */ + public enum RequestFollowUpPlan { + /** Aggressive follow-up with shorter intervals */ + SPEEDY, + /** Relaxed follow-up with longer intervals */ + SLOW + } +} +``` + +### Step 5: Create Repository + +```java +package no.offpost.repository; + +import no.offpost.domain.Thread; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Query; +import org.springframework.stereotype.Repository; +import java.util.List; + +/** + * Repository interface for managing Thread entities. + * + *

Provides data access operations for email threads, including custom queries + * for retrieving threads by various criteria such as archived status, entity association, + * and sending status. + * + * @author Offpost Team + * @see Thread + */ +@Repository +public interface ThreadRepository extends JpaRepository { + + /** + * Finds all non-archived threads ordered by last update time (most recent first). + * + * @return list of threads that are not archived, ordered by updatedAt descending + */ + List findByArchivedFalseOrderByUpdatedAtDesc(); + + /** + * Finds all non-archived threads for a specific entity. + * + * @param entityId the ID of the public entity + * @return list of non-archived threads associated with the specified entity + */ + List findByEntityIdAndArchivedFalse(Integer entityId); + + /** + * Finds all threads that are ready to be sent, ordered by creation time (oldest first). + * + *

This method is used by the scheduled email sender to retrieve threads + * awaiting delivery to public entities. + * + * @return list of threads with READY_FOR_SENDING status, ordered by creation time ascending + */ + @Query("SELECT t FROM Thread t WHERE t.sendingStatus = 'READY_FOR_SENDING' ORDER BY t.createdAt ASC") + List findReadyForSending(); +} +``` + +### Step 6: Create Basic REST Controller + +```java +package no.offpost.controller; + +import no.offpost.domain.Thread; +import no.offpost.repository.ThreadRepository; +import org.springframework.http.ResponseEntity; +import org.springframework.web.bind.annotation.*; +import lombok.RequiredArgsConstructor; +import java.util.List; + +/** + * REST controller for managing email threads. + * + *

Provides HTTP endpoints for CRUD operations on Thread entities. + * All endpoints are prefixed with {@code /api/threads}. + * + * @author Offpost Team + * @see Thread + * @see ThreadRepository + */ +@RestController +@RequestMapping("/api/threads") +@RequiredArgsConstructor +public class ThreadController { + + private final ThreadRepository threadRepository; + + /** + * Retrieves all non-archived threads. + * + *

Returns threads ordered by last update time, with most recently updated threads first. + * + * @return list of all non-archived threads + */ + @GetMapping + public List getAllThreads() { + return threadRepository.findByArchivedFalseOrderByUpdatedAtDesc(); + } + + /** + * Retrieves a specific thread by its ID. + * + * @param id the unique identifier of the thread + * @return HTTP 200 with the thread if found, HTTP 404 if not found + */ + @GetMapping("/{id}") + public ResponseEntity getThread(@PathVariable String id) { + return threadRepository.findById(id) + .map(ResponseEntity::ok) + .orElse(ResponseEntity.notFound().build()); + } + + /** + * Creates a new thread. + * + *

The thread will be automatically assigned a UUID and timestamps. + * Initial status will be set to STAGING if not specified. + * + * @param thread the thread to create + * @return the created thread with generated ID and timestamps + */ + @PostMapping + public Thread createThread(@RequestBody Thread thread) { + return threadRepository.save(thread); + } + + /** + * Updates an existing thread. + * + *

Replaces the thread with the given ID with the provided thread data. + * The updatedAt timestamp will be automatically updated. + * + * @param id the unique identifier of the thread to update + * @param thread the updated thread data + * @return HTTP 200 with the updated thread if found, HTTP 404 if not found + */ + @PutMapping("/{id}") + public ResponseEntity updateThread( + @PathVariable String id, + @RequestBody Thread thread) { + if (!threadRepository.existsById(id)) { + return ResponseEntity.notFound().build(); + } + thread.setId(id); + return ResponseEntity.ok(threadRepository.save(thread)); + } +} +``` + +### Step 7: Copy SQL Migrations + +Copy the SQL migrations from PHP project: + +```bash +# From project root +mkdir -p offpost-java/src/main/resources/db/migration +cp organizer/src/migrations/sql/*.sql offpost-java/src/main/resources/db/migration/ + +# Rename to Flyway format +cd offpost-java/src/main/resources/db/migration/ +for file in *.sql; do + # Extract number from filename and pad with zeros + num=$(echo "$file" | grep -o '[0-9]*' | head -1) + padded=$(printf "%03d" $num) + # Rename to Flyway format: V###__description.sql + newname=$(echo "$file" | sed "s/^[0-9]*-/V${padded}__/") + mv "$file" "$newname" +done +``` + +### Step 8: Create Dockerfile + +```dockerfile +# Build stage +FROM maven:3.9-eclipse-temurin-21 AS build +WORKDIR /app +COPY pom.xml . +COPY src ./src +RUN mvn clean package -DskipTests + +# Runtime stage +FROM eclipse-temurin:21-jre +WORKDIR /app +COPY --from=build /app/target/*.jar app.jar +EXPOSE 8080 +ENTRYPOINT ["java", "-jar", "app.jar"] +``` + +### Step 9: Create Docker Compose for Development + +Create `docker-compose.yml` in Java project: + +```yaml +version: '3.8' + +services: + postgres: + image: postgres:15 + environment: + POSTGRES_DB: offpost + POSTGRES_USER: offpost + POSTGRES_PASSWORD: offpost_dev + ports: + - "5432:5432" + volumes: + - postgres_data:/var/lib/postgresql/data + + greenmail: + image: greenmail/standalone:2.1.3 + ports: + - "3025:3025" # SMTP + - "3143:3143" # IMAP + - "3993:3993" # IMAPS + - "8080:8080" # Web UI + environment: + - GREENMAIL_OPTS=-Dgreenmail.setup.test.all -Dgreenmail.hostname=0.0.0.0 -Dgreenmail.users=greenmail-user:password@dev.offpost.no + + app: + build: . + ports: + - "8081:8080" + depends_on: + - postgres + - greenmail + environment: + - SPRING_DATASOURCE_URL=jdbc:postgresql://postgres:5432/offpost + - SPRING_DATASOURCE_USERNAME=offpost + - SPRING_DATASOURCE_PASSWORD=offpost_dev + - IMAP_SERVER=greenmail + - IMAP_PORT=3993 + - IMAP_EMAIL=greenmail-user@dev.offpost.no + - IMAP_PASSWORD=password + +volumes: + postgres_data: +``` + +### Step 10: Test the Setup + +```bash +# Build and run +mvn clean install +mvn spring-boot:run + +# Or with Docker +docker-compose up --build + +# Test the API +curl http://localhost:8080/api/threads +curl http://localhost:8080/actuator/health +``` + +## Next Steps + +1. **Port Domain Models**: Complete all entity classes (Phase 2) +2. **Port Repositories**: Add remaining repository interfaces (Phase 3) +3. **IMAP Integration**: Implement email handling (Phase 4) +4. **Unit Tests**: Start porting tests alongside code +5. **Follow Migration Plan**: See MIGRATION_PLAN_PHP_TO_JAVA.md for detailed phases + +## Common Issues + +### Issue: Lombok not working in IDE +**Solution**: Install Lombok plugin and enable annotation processing + +### Issue: Flyway migration fails +**Solution**: Check migration file naming (V###__description.sql) and ensure sequential versioning + +### Issue: Cannot connect to PostgreSQL +**Solution**: Verify Docker container is running and port 5432 is not in use + +## Resources + +- [Spring Boot Documentation](https://spring.io/projects/spring-boot) +- [Spring Data JPA](https://spring.io/projects/spring-data-jpa) +- [JavaMail API](https://javaee.github.io/javamail/) +- [Flyway](https://flywaydb.org/documentation/) +- [Lombok](https://projectlombok.org/) + +## Getting Help + +- Review MIGRATION_PLAN_PHP_TO_JAVA.md for comprehensive migration strategy +- Check Spring Boot documentation for framework questions +- Refer to original PHP code for business logic reference diff --git a/docs/migration/PHP_TO_JAVA_MAPPING.md b/docs/migration/PHP_TO_JAVA_MAPPING.md new file mode 100644 index 0000000..4d214cb --- /dev/null +++ b/docs/migration/PHP_TO_JAVA_MAPPING.md @@ -0,0 +1,267 @@ +# PHP to Java Migration Mapping + +This document provides a detailed mapping of PHP files to their Java equivalents. + +## Directory Structure Mapping + +| PHP Location | Java Location | Notes | +|--------------|---------------|-------| +| `organizer/src/class/` | `src/main/java/no/offpost/domain/` and `src/main/java/no/offpost/service/` | Split between entities and services | +| `organizer/src/api/` | `src/main/java/no/offpost/controller/` | REST controllers | +| `organizer/src/system-pages/` | `src/main/java/no/offpost/scheduler/` | Spring @Scheduled tasks | +| `organizer/src/tests/` | `src/test/java/no/offpost/unit/` | JUnit 5 tests | +| `organizer/src/e2e-tests/` | `src/test/java/no/offpost/e2e/` | Integration tests | +| `organizer/src/migrations/sql/` | `src/main/resources/db/migration/` | Flyway migrations | +| `organizer/src/*.php` (UI pages) | `src/main/resources/templates/` | Thymeleaf templates + controllers | + +## Core Domain Classes + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/Thread.php` | `no.offpost.domain` | `Thread.java` | JPA entity with relationships | +| `class/ThreadEmail.php` | `no.offpost.domain` | `ThreadEmail.java` | JPA entity | +| `class/ThreadEmailAttachment.php` | `no.offpost.domain` | `ThreadEmailAttachment.java` | JPA entity | +| `class/Entity.php` | `no.offpost.domain` | `Entity.java` | JPA entity for public organizations | +| `class/Identity.php` | `no.offpost.domain` | `Identity.java` | JPA entity | +| `class/ImapFolderStatus.php` | `no.offpost.domain` | `ImapFolderStatus.java` | JPA entity | +| `class/ImapFolderLog.php` | `no.offpost.domain` | `ImapFolderLog.java` | JPA entity | +| `class/ThreadHistory.php` | `no.offpost.domain` | `ThreadHistory.java` | JPA entity | + +## Enums + +| PHP Constants | Java Package | Java Enum | Values | +|---------------|--------------|-----------|--------| +| `Thread::SENDING_STATUS_*` | `no.offpost.domain` | `Thread.SendingStatus` | STAGING, READY_FOR_SENDING, SENDING, SENT | +| `Thread::REQUEST_LAW_BASIS_*` | `no.offpost.domain` | `Thread.RequestLawBasis` | OFFENTLEGLOVA, OTHER | +| `Thread::REQUEST_FOLLOW_UP_PLAN_*` | `no.offpost.domain` | `Thread.RequestFollowUpPlan` | SPEEDY, SLOW | + +## Database Layer + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/Database.php` | `org.springframework.boot.autoconfigure.jdbc` | (Spring Boot auto-config) | Replaced by Spring Data JPA configuration | +| `class/ThreadDatabaseOperations.php` | `no.offpost.service` | `ThreadService.java` | Business logic with repository injection | +| `class/ThreadStatusRepository.php` | `no.offpost.repository` | `ThreadRepository.java` with custom queries | JPA repository with @Query annotations | +| `class/ThreadStorageManager.php` | `no.offpost.service` | `ThreadStorageService.java` | Service layer, no longer singleton | + +## Repository Interfaces (New in Java) + +| PHP Context | Java Package | Java Interface | Extends | +|-------------|--------------|----------------|---------| +| Thread operations | `no.offpost.repository` | `ThreadRepository` | `JpaRepository` | +| ThreadEmail operations | `no.offpost.repository` | `ThreadEmailRepository` | `JpaRepository` | +| Entity operations | `no.offpost.repository` | `EntityRepository` | `JpaRepository` | +| Identity operations | `no.offpost.repository` | `IdentityRepository` | `JpaRepository` | +| ImapFolderStatus operations | `no.offpost.repository` | `ImapFolderStatusRepository` | `JpaRepository` | +| ImapFolderLog operations | `no.offpost.repository` | `ImapFolderLogRepository` | `JpaRepository` | + +## IMAP Integration + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/Imap/ImapWrapper.php` | `no.offpost.service.imap` | `ImapService.java` | Uses JavaMail API | +| `class/Imap/ImapConnection.php` | `no.offpost.service.imap` | `ImapConnectionManager.java` | Connection pooling with JavaMail | +| `class/Imap/ImapFolderManager.php` | `no.offpost.service.imap` | `ImapFolderService.java` | Folder operations | +| `class/Imap/ImapEmail.php` | `no.offpost.service.imap` | `ImapEmailParser.java` | Email parsing | +| `class/Imap/ImapEmailProcessor.php` | `no.offpost.service.imap` | `ImapEmailProcessor.java` | Email processing logic | +| `class/Imap/ImapAttachmentHandler.php` | `no.offpost.service.imap` | `ImapAttachmentHandler.java` | Attachment handling | +| `class/ThreadEmailMover.php` | `no.offpost.service.email` | `ThreadEmailMoverService.java` | Email moving operations | +| `class/ThreadFolderManager.php` | `no.offpost.service.email` | `ThreadFolderService.java` | Thread folder management | + +## Email Processing Services + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/ThreadScheduledEmailReceiver.php` | `no.offpost.scheduler` | `EmailReceiverScheduler.java` | @Scheduled annotation | +| `class/ThreadScheduledEmailSender.php` | `no.offpost.scheduler` | `EmailSenderScheduler.java` | @Scheduled annotation | +| `class/ThreadScheduledFollowUpSender.php` | `no.offpost.scheduler` | `FollowUpScheduler.java` | @Scheduled annotation | +| `class/ThreadEmailDatabaseSaver.php` | `no.offpost.service.email` | `EmailDatabaseService.java` | EML parsing and saving | +| `class/ThreadEmailSending.php` | `no.offpost.service.email` | `EmailSendingService.java` | SendGrid integration | +| `class/ThreadEmailService.php` | `no.offpost.service.email` | `ThreadEmailService.java` | Email operations | +| `class/ThreadEmailHistory.php` | `no.offpost.service.email` | `EmailHistoryService.java` | Email history tracking | +| `class/ThreadEmailProcessingErrorManager.php` | `no.offpost.service.email` | `EmailErrorService.java` | Error handling | + +## AI Integration + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/Ai/*.php` | `no.offpost.service.ai` | Various AI service classes | OpenAI integration | +| `class/SuggestedReplyGenerator.php` | `no.offpost.service.ai` | `ReplyGeneratorService.java` | GPT-based reply generation | +| `class/ThreadEmailClassifier.php` | `no.offpost.service.ai` | `EmailClassifierService.java` | Email classification | +| `class/Extraction/*.php` | `no.offpost.service.extraction` | Various extraction services | Content extraction (PDF, images, etc.) | + +## REST API Controllers + +| PHP File | Java Package | Java Class(es) | HTTP Methods | +|----------|--------------|----------------|--------------| +| `api/thread_email_extraction.php` | `no.offpost.controller` | `ExtractionController.java` | POST /api/extraction | +| Thread API operations | `no.offpost.controller` | `ThreadController.java` | GET, POST, PUT, DELETE /api/threads | +| Email operations | `no.offpost.controller` | `EmailController.java` | Various /api/emails endpoints | +| Entity operations | `no.offpost.controller` | `EntityController.java` | Various /api/entities endpoints | +| Admin operations | `no.offpost.controller` | `AdminController.java` | Various /api/admin endpoints | + +## Scheduled Task Endpoints + +| PHP File | Java Package | Java Class(es) | Trigger | +|----------|--------------|----------------|---------| +| `system-pages/scheduled-email-sending.php` | `no.offpost.scheduler` | `EmailSenderScheduler.java` | @Scheduled(cron = "...") | +| `system-pages/scheduled-email-receiver.php` | `no.offpost.scheduler` | `EmailReceiverScheduler.java` | @Scheduled(fixedDelay = ...) | +| `system-pages/scheduled-imap-handling.php` | `no.offpost.scheduler` | `ImapMaintenanceScheduler.java` | @Scheduled(cron = "...") | +| `system-pages/scheduled-email-extraction.php` | `no.offpost.scheduler` | `ExtractionScheduler.java` | @Scheduled(cron = "...") | +| `system-pages/scheduled-thread-follow-up.php` | `no.offpost.scheduler` | `FollowUpScheduler.java` | @Scheduled(cron = "...") | + +## Authentication and Authorization + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `auth.php` | `no.offpost.config` | `SecurityConfig.java` | Spring Security configuration | +| `callback.php` | `no.offpost.controller` | `AuthController.java` | Auth0 callback handler | +| `class/ThreadAuthorization.php` | `no.offpost.service.security` | `ThreadAuthorizationService.java` | Permission checks | +| `username-password.php` | (Configuration) | `application.yml` or environment variables | Credentials configuration | + +## Utility Classes + +| PHP File | Java Package | Java Class(es) | Notes | +|----------|--------------|----------------|-------| +| `class/common.php` | `no.offpost.util` | Various utility classes | Common functions split into appropriate utilities | +| `class/ThreadUtils.php` | `no.offpost.util` | `ThreadUtils.java` | Thread-related utilities | +| `class/ThreadLabelFilter.php` | `no.offpost.util` | `LabelFilterUtil.java` | Label filtering logic | +| `class/random-profile.php` | `no.offpost.util` | `ProfileGenerator.java` | Random profile generation | +| `class/AdminNotificationService.php` | `no.offpost.service` | `AdminNotificationService.java` | Admin notifications | + +## Web UI Pages (Option: Thymeleaf) + +| PHP File | Java Controller | Template | Notes | +|----------|-----------------|----------|-------| +| `index.php` | `HomeController.java` | `templates/index.html` | Thread listing | +| `view-thread.php` | `ThreadViewController.java` | `templates/thread/view.html` | Thread details | +| `start-thread.php` | `ThreadController.java` | `templates/thread/create.html` | Thread creation form | +| `thread-reply.php` | `ThreadReplyController.java` | `templates/thread/reply.html` | Reply form | +| `entities.php` | `EntityController.java` | `templates/entity/list.html` | Entity management | +| `thread-bulk-actions.php` | `BulkActionsController.java` | `templates/thread/bulk.html` | Bulk operations | +| `update-identities.php` | `MaintenanceController.java` | `templates/maintenance/identities.html` | Identity updates | +| `update-imap.php` | `MaintenanceController.java` | `templates/maintenance/imap.html` | IMAP sync | +| `recent-activity.php` | `ActivityController.java` | `templates/activity/recent.html` | Activity log | +| `grant-thread-access.php` | `AccessController.java` | `templates/access/grant.html` | Access management | + +## Test Files + +| PHP Test File | Java Test Package | Java Test Class | Framework | +|---------------|-------------------|-----------------|-----------| +| `tests/ThreadScheduledEmailSenderTest.php` | `no.offpost.scheduler` | `EmailSenderSchedulerTest.java` | JUnit 5 + Mockito | +| `tests/ImapWrapperRetryTest.php` | `no.offpost.service.imap` | `ImapServiceRetryTest.java` | JUnit 5 + TestContainers | +| `tests/ThreadHistoryTest.php` | `no.offpost.service` | `ThreadHistoryServiceTest.java` | JUnit 5 | +| `tests/ThreadEmailHeaderProcessingTest.php` | `no.offpost.service.email` | `EmailHeaderProcessingTest.java` | JUnit 5 | +| `tests/Ai/OpenAiIntegrationTest.php` | `no.offpost.service.ai` | `OpenAiIntegrationTest.java` | JUnit 5 + WireMock | +| `e2e-tests/pages/*Test.php` | `no.offpost.e2e` | Various E2E test classes | JUnit 5 + Selenium/TestContainers | + +## Configuration Files + +| PHP/Current | Java/Spring Boot | Notes | +|-------------|------------------|-------| +| `composer.json` | `pom.xml` | Dependency management | +| `docker-compose.dev.yaml` | `docker-compose.yml` (updated) | Development environment | +| `docker-compose.prod.yaml` | `docker-compose.prod.yml` (updated) | Production environment | +| `.env` files | `application.yml` + environment variables | Configuration | +| PHP include paths | Spring component scanning | Automatic in Spring | + +## Database Migrations + +| PHP Migration | Flyway Migration | Notes | +|---------------|------------------|-------| +| `migrations/sql/00001-*.sql` | `db/migration/V001__*.sql` | Rename with Flyway convention | +| `migrations/sql/00002-*.sql` | `db/migration/V002__*.sql` | Sequential versioning | +| ... | ... | ... | +| `migrations/sql/00030-*.sql` | `db/migration/V030__*.sql` | All 30 migrations | +| `migrations/sql/99999-database-schema-after-migrations.sql` | (Generated by Flyway) | Schema documentation | + +## PHP Function Mappings + +### Common PHP → Java Equivalents + +| PHP | Java | Notes | +|-----|------|-------| +| `require_once` | `import` | Java imports | +| `array()` or `[]` | `new ArrayList<>()` or `List.of()` | Collections | +| `json_encode()` | `ObjectMapper.writeValueAsString()` | Jackson library | +| `json_decode()` | `ObjectMapper.readValue()` | Jackson library | +| `file_get_contents()` | `Files.readString()` | Java NIO | +| `file_put_contents()` | `Files.writeString()` | Java NIO | +| `mt_rand()` | `Random.nextInt()` | Avoid in tests! | +| `time()` | `Instant.now().getEpochSecond()` | Avoid in tests! | +| `date()` | `LocalDateTime.format()` | Java Time API | +| `var_dump()` | `System.out.println()` or logger | Debugging | +| `die()` / `exit()` | `throw new RuntimeException()` | Error handling | +| `isset()` | `!= null` or `Optional.ofNullable()` | Null checking | +| `empty()` | `== null || isEmpty()` | Null/empty checking | +| `$_GET` / `$_POST` | `@RequestParam` / `@RequestBody` | Spring annotations | +| `$_SESSION` | `HttpSession` or JWT | Session management | +| PDO prepared statements | JPA/JDBC | ORM or prepared statements | + +## Library Mappings + +| PHP Library | Java Library | Purpose | +|-------------|--------------|---------| +| PHPMailer | JavaMail API | Email sending | +| Laminas Mail | JavaMail API | Email parsing | +| PDO | Spring Data JPA / JDBC | Database access | +| cURL | RestTemplate / WebClient | HTTP client | +| OpenSSL | Java Crypto API | Encryption | +| PHPUnit | JUnit 5 + Mockito | Testing | +| Composer | Maven | Dependency management | + +## Naming Convention Changes + +| PHP Convention | Java Convention | Example | +|----------------|-----------------|---------| +| snake_case variables | camelCase | `my_email` → `myEmail` | +| snake_case methods | camelCase | `get_thread_by_id()` → `getThreadById()` | +| PascalCase classes | PascalCase | Same: `ThreadEmail` | +| UPPER_CASE constants | UPPER_CASE | Same: `SENDING_STATUS_SENT` | +| snake_case DB columns | snake_case | Same (JPA @Column mapping) | +| `function` keyword | Method in class | All code in classes | + +## Architecture Changes + +| PHP Pattern | Java/Spring Pattern | Benefit | +|-------------|---------------------|---------| +| Singleton classes | Spring @Service beans | Automatic dependency injection | +| `require_once` dependencies | Constructor injection | Better testability | +| Global functions | Static utility methods | Better organization | +| Mixed SQL and logic | Repository pattern | Separation of concerns | +| Cron-triggered scripts | @Scheduled annotations | Application-managed scheduling | +| Session-based auth | JWT or OAuth2 | Stateless authentication | +| Mixed HTML/PHP | Thymeleaf or REST+SPA | Separation of presentation | + +## Key Differences to Remember + +1. **Type Safety**: Java is statically typed; all variables need explicit types +2. **Null Handling**: Java has `null`, consider using `Optional` for nullable values +3. **Error Handling**: Use exceptions instead of return values for errors +4. **Collections**: Generic types required: `List` not just `List` +5. **Package Structure**: Deep package hierarchy is standard in Java +6. **Testing**: Test classes in mirror package structure under `src/test/java` +7. **Configuration**: Properties in `application.yml` instead of `.env` files +8. **Dependency Injection**: Constructor injection preferred over field injection +9. **Annotations**: Heavy use of annotations for configuration (@Entity, @Service, @RestController) +10. **Build Process**: Maven handles compilation, packaging, and dependencies + +## Implementation Priority + +Recommended order for migration: + +1. **Domain Entities** (Phase 2) - Start here +2. **Repository Layer** (Phase 3) - Enable data access +3. **Service Layer** (Parts of Phases 4-6) - Core business logic +4. **REST API** (Phase 7) - Enable frontend integration +5. **Scheduled Tasks** (Part of Phase 5) - Background processing +6. **Web UI** (Phase 8) - User interface +7. **Tests** (Phase 9) - Throughout all phases + +## Getting Help + +- For specific class ports, refer to the original PHP file for business logic +- Spring Boot documentation covers most framework questions +- JavaMail API documentation for IMAP operations +- JUnit 5 documentation for test porting +- Consult MIGRATION_PLAN_PHP_TO_JAVA.md for overall strategy diff --git a/docs/migration/README.md b/docs/migration/README.md new file mode 100644 index 0000000..0e10612 --- /dev/null +++ b/docs/migration/README.md @@ -0,0 +1,235 @@ +# PHP to Java Migration - Executive Summary + +## Overview + +This repository now contains a comprehensive plan for migrating the Offpost email thread management system from PHP to Java with Spring Boot. This document serves as the entry point to all migration planning materials. + +## Current Status + +**✓ Planning Phase Complete** - Ready for stakeholder review and implementation + +## What We're Migrating + +**From**: PHP 8.x application with 181 files +- Organizer component (main application) +- 57 core business logic classes +- IMAP email handling +- OpenAI integration for content extraction +- PostgreSQL database (30 migrations) +- REST APIs and web UI + +**To**: Java 21 + Spring Boot 3.2+ application +- Modern, maintainable enterprise architecture +- Spring Data JPA for database +- JavaMail API for IMAP +- Spring Security with Auth0 +- RESTful APIs with OpenAPI documentation +- Testable, scalable design + +## Why Java? + +1. **Type Safety**: Catch errors at compile time +2. **Performance**: Better performance for background processing +3. **Scalability**: Easier to scale horizontally +4. **Maintainability**: Better IDE support and refactoring tools +5. **Ecosystem**: Rich Spring Boot ecosystem for enterprise features +6. **Testing**: Strong testing frameworks and tools +7. **Community**: Large enterprise Java community and resources + +## Documentation Structure + +### 📋 Main Planning Document +**[MIGRATION_PLAN_PHP_TO_JAVA.md](./MIGRATION_PLAN_PHP_TO_JAVA.md)** (22,000+ words) +- Executive summary +- 11-phase migration plan (26 weeks) +- Detailed task breakdowns for each phase +- Key technical decisions (Framework, Build Tool, Libraries) +- Risk analysis and mitigation strategies +- Success criteria +- Timeline and resource estimates + +### 🚀 Quick Start Guide +**[docs/migration/GETTING_STARTED_JAVA.md](./docs/migration/GETTING_STARTED_JAVA.md)** +- Prerequisites and setup +- Step-by-step Spring Boot project creation +- Sample code for domain model, repository, and REST API +- Docker Compose configuration +- Common issues and solutions +- Immediate next steps + +### 🗺️ File Mapping Reference +**[docs/migration/PHP_TO_JAVA_MAPPING.md](./docs/migration/PHP_TO_JAVA_MAPPING.md)** +- Complete mapping of 181 PHP files to Java equivalents +- Directory structure transformation +- Class-by-class conversion guide +- PHP function to Java method mappings +- Library equivalents (PHPMailer → JavaMail, PDO → JPA) +- Test file migration mapping +- Configuration file changes + +## Migration Phases at a Glance + +| Phase | Duration | Focus Area | Key Deliverables | +|-------|----------|------------|------------------| +| 1 | 2 weeks | Foundation & Setup | Spring Boot project, Docker config | +| 2 | 2 weeks | Core Domain Model | JPA entities, enums | +| 3 | 2 weeks | Database Layer | Flyway migrations, repositories | +| 4 | 3 weeks | IMAP Integration | JavaMail implementation, retry logic | +| 5 | 3 weeks | Email Processing | Scheduled tasks, SendGrid | +| 6 | 2 weeks | AI Integration | OpenAI client, extraction services | +| 7 | 2 weeks | REST API Layer | Controllers, Auth0 integration | +| 8 | 4 weeks | Web Frontend | Thymeleaf or SPA | +| 9 | 2 weeks | Testing Infrastructure | JUnit 5, TestContainers | +| 10 | 2 weeks | Deployment | Docker, K8s, monitoring | +| 11 | 2 weeks | Documentation | Updated docs, runbooks | +| **Total** | **26 weeks** | **Complete Migration** | **Production-ready Java app** | + +## Key Technical Decisions + +### ✅ Framework: Spring Boot 3.2+ +- Industry standard for enterprise Java applications +- Comprehensive ecosystem (Web, Data, Security, Cloud) +- Excellent documentation and community support +- Built-in support for REST APIs, JPA, scheduling + +### ✅ Build Tool: Maven 3.9+ +- More widely used in enterprise Java +- Better IDE support +- Simpler for standard Spring Boot projects + +### ✅ Database Migration: Flyway +- SQL-based approach (easy migration from existing SQL) +- Good Spring Boot integration +- Version-based migration tracking + +### ✅ Testing: JUnit 5 + Mockito + TestContainers +- Modern testing framework +- Integration tests with real databases (TestContainers) +- GreenMail for email testing + +### ✅ Frontend: Thymeleaf (MVP), then SPA +- Server-side rendering for quick feature parity +- Can transition to React/Vue incrementally + +## Migration Strategy + +### 🎯 Approach: Incremental Migration +1. Build Java version alongside PHP (not big bang) +2. Start with API and core domain +3. Match existing functionality before enhancements +4. Port tests early to validate equivalence +5. Use feature flags during transition +6. Gradual traffic shift (canary/blue-green deployment) + +### 🛡️ Risk Mitigation +- Comprehensive testing at every phase +- Parallel runs to compare behavior +- Database migration scripts with rollback +- Backup procedures +- Performance monitoring from day one + +## Files and Scope + +### Core Components to Port +- **57 class files** in `organizer/src/class/` +- **30 SQL migrations** in `migrations/sql/` +- **API endpoints** in `api/` +- **Scheduled tasks** in `system-pages/` +- **Unit tests** in `tests/` +- **E2E tests** in `e2e-tests/` +- **Web pages** (index, view-thread, etc.) + +### External Integrations to Maintain +- **PostgreSQL 15** - Main database +- **MySQL 5.6** - Roundcube (unchanged) +- **IMAP Server** - GreenMail (dev), production server +- **SendGrid** - Email delivery +- **OpenAI API** - Content extraction and AI features +- **Auth0** - Authentication (production) + +## Success Criteria + +1. ✅ **Functional Parity**: All PHP features working in Java +2. ✅ **Test Coverage**: >80% with passing tests +3. ✅ **Performance**: Response times ≤ PHP version +4. ✅ **Stability**: Zero critical bugs in first month +5. ✅ **Documentation**: Complete and accurate +6. ✅ **Deployment**: Successful production deployment with rollback capability + +## Resource Requirements + +- **2-3 Java Developers** (full-time) +- **1 QA Engineer** (full-time) +- **0.5 DevOps Engineer** (part-time) +- **0.25 Technical Writer** (part-time) + +## Timeline + +**Estimated Duration**: 26 weeks (6.5 months) + +Assumes: +- Full-time dedicated resources +- Team familiar with Java/Spring Boot +- Minimal scope changes +- Parallel work where possible + +## Next Steps + +### For Stakeholders +1. **Review** all planning documents +2. **Approve** migration approach and timeline +3. **Allocate** resources (developers, QA, DevOps) +4. **Set** project kickoff date + +### For Development Team +1. **Read** MIGRATION_PLAN_PHP_TO_JAVA.md thoroughly +2. **Follow** GETTING_STARTED_JAVA.md to set up environment +3. **Reference** PHP_TO_JAVA_MAPPING.md during implementation +4. **Start** with Phase 1: Foundation and Setup + +### Immediate Actions +```bash +# 1. Set up Java development environment +# Install Java 21 LTS, Maven 3.9+, Docker + +# 2. Create Spring Boot project +# Follow docs/migration/GETTING_STARTED_JAVA.md + +# 3. Review current PHP codebase +cd organizer/src +find class/ -name "*.php" | head -20 # Explore classes + +# 4. Start with domain model (Phase 2) +# Port Thread.php to Thread.java first +``` + +## Questions? + +- **Strategic questions**: See MIGRATION_PLAN_PHP_TO_JAVA.md Risk Analysis section +- **Technical questions**: See PHP_TO_JAVA_MAPPING.md for specific conversions +- **Getting started**: See GETTING_STARTED_JAVA.md for step-by-step guide + +## Contributing + +When implementing the migration: +1. Follow the phase sequence in the migration plan +2. Port tests alongside code +3. Use the PHP-to-Java mapping as reference +4. Maintain feature parity before enhancements +5. Document any deviations from the plan + +## Conclusion + +This comprehensive planning effort provides a clear roadmap for migrating from PHP to Java. The plan is: +- **Detailed**: Every PHP file mapped to Java equivalent +- **Actionable**: Step-by-step implementation guide +- **Risk-aware**: Identified risks with mitigation strategies +- **Realistic**: 26-week timeline with resource requirements +- **Testable**: Testing strategy throughout + +**The migration is now ready to proceed from planning to implementation.** + +--- + +*Planning completed: 2026-02-06* +*Ready for: Stakeholder review and Phase 1 kickoff*