formies/notes.md

## Task 2.1: User Dashboard & Form Management UI (Replacing current "admin")

- Mindset Shift: This is no longer an admin panel. It's the user's control center.

### Subtask 2.1.1: Design User Dashboard Layout

- **Wireframe basic layout:**
  - **Navigation Bar:**
    - Logo/App Name (e.g., "Formies")
    - My Forms (Active Link)
    - Create New Form
    - Account Settings (e.g., "Hi, [User Name]" dropdown with "Settings", "Logout")
  - **Main Content Area (for "My Forms" view):**
    - Header: "My Forms"
    - Button: "+ Create New Form"
    - Forms List Table:
      - Columns: Form Name, Submissions (count), Endpoint URL, Created Date, Actions
      - Actions per row: View Submissions, Settings, Archive/Delete
    - Pagination for the forms list if it becomes long.
  - **Main Content Area (for "Create New Form" view - initial thought, might be a separate page/modal):**
    - Header: "Create New Form"
    - Form fields: Form Name
    - Button: "Create Form"
  - **Main Content Area (for "Account Settings" - placeholder for now):**
    - Header: "Account Settings"
    - Placeholder content.
- **Frontend Tech Decision:**
  - EJS for templating, made dynamic with client-side JavaScript. This aligns with the existing structure and MVP scope. We will enhance EJS views to be more interactive.

[X] Wireframe basic layout: List forms, create form, account settings (placeholder). - _Textual wireframe defined above_
[X] Decide on frontend tech (EJS is fine for MVP if you make it dynamic with client-side JS, or consider a simple SPA framework if comfortable). - _Decision made: EJS with client-side JS_

- Created `views/dashboard.ejs` as the main layout.
- Created `views/partials/_forms_table.ejs` for displaying the list of forms.

### Subtask 2.1.2: "My Forms" View:

- Objective: Fetch and display forms owned by the logged-in user.
- Show key info: name, submission count, endpoint URL, created date, status (Active/Archived).
- Links/Actions: View Submissions, Settings, Archive/Unarchive, Delete.
- Frontend: `views/dashboard.ejs` with `view = 'my_forms'` and `views/partials/_forms_table.ejs` will handle this.
- Backend:
  - Need a new route, e.g., `GET /dashboard`, protected by authentication (e.g., `requireAuth` from `authMiddleware.js`).
  - This route handler will:
    - Fetch forms for `req.user.id` from the database.
    - Query should include `name`, `uuid`, `created_at`, `is_archived`, and `submission_count`.
    - Render `views/dashboard.ejs` passing the forms data, `user` object, `appUrl`, and `view = 'my_forms'`.
  - Implemented in `src/routes/dashboard.js` via GET `/`.

[X] Fetch and display forms owned by the logged-in user.
[X] Show key info: name, submission count, endpoint URL, created date.
[X] Links to: view submissions, edit settings, delete. (Links are present in `_forms_table.ejs`, functionality for all to be built out in subsequent tasks)

### Subtask 2.1.3: "Create New Form" Functionality (for logged-in user):

- UI: `dashboard.ejs` (with `view = 'create_form'`) provides the form input.
  - Route `GET /dashboard/create-form` in `src/routes/dashboard.js` renders this view.
- Backend: `POST /dashboard/forms/create` route in `src/routes/dashboard.js` handles form submission.
  - Associates form with `req.user.id`.
  - Redirects to `/dashboard` on success.
  - Handles errors and re-renders create form view with an error message.

[X] UI and backend logic. Associates form with req.user.id.

### Subtask 2.1.4: "View Form Submissions" (Scoped & Paginated):

- Objective: Allow users to view submissions for their specific forms, with pagination.
- UI:
  - `views/partials/_submissions_view.ejs` created to display submissions list and pagination.
  - `views/dashboard.ejs` updated to include this partial when `view = 'form_submissions'`.
- Backend:
  - Route: `GET /dashboard/submissions/:formUuid` added to `src/routes/dashboard.js`.
  - Verifies that `req.user.id` owns the `formUuid`.
  - Fetches paginated submissions for the given `formUuid`.
  - Renders `dashboard.ejs` with `view = 'form_submissions'`, passing submissions data, form details, and pagination info.
  - Error handling improved to render user-friendly messages within the dashboard view.

[X] UI and backend for a user to view submissions for their specific form.
[X] Pagination is critical here (as you have).

### Subtask 2.1.5: Form Settings UI (Basic):

- Objective: Allow users to update basic form settings, starting with the form name.
- UI:
  - A new view/section in `dashboard.ejs` (e.g., when `view = 'form_settings'`).
  - This view will display a form with an input for the form name.
  - It will also be a placeholder for future settings (thank you URL, notifications).
- Backend:
  - Route: `GET /dashboard/forms/:formUuid/settings` to display the settings page.
    - Implemented in `src/routes/dashboard.js`.
    - Verifies form ownership by `req.user.id`.
    - Fetches current form details (name).
    - Renders the `form_settings` view in `dashboard.ejs`.
  - Route: `POST /dashboard/forms/:formUuid/settings/update-name` to handle the update.
    - Implemented in `src/routes/dashboard.js`.
    - Verifies form ownership.
    - Updates the form name in the database.
    - Redirects back to form settings page with a success/error message via query parameters.

[X] Allow users to update form name.
[X] Placeholder for future settings (thank you URL, notifications) - (Placeholders added in EJS).

### Subtask 2.1.6: Delete Form/Submission (with soft delete/archival consideration):

- Objective: Implement form archival (soft delete) and permanent deletion for users.
- Users should be able to archive/unarchive their forms.
- True delete should be a confirmed, rare operation.
- The `is_archived` field in the `forms` table will be used.
- Submissions deletion is already partially handled in `_submissions_view.ejs` via a POST to `/dashboard/submissions/delete/:submissionId`. We need to implement this backend route.

- **Form Archival/Unarchival:**
  - UI: Buttons for "Archive" / "Unarchive" are already in `views/partials/_forms_table.ejs`.
    - Archive action: `POST /dashboard/forms/archive/:formUuid`
    - Unarchive action: `POST /dashboard/forms/unarchive/:formUuid`
  - Backend:
    - Create these two POST routes in `src/routes/dashboard.js`.
    - Must verify form ownership by `req.user.id`.
    - Fetch current form details (name).
    - Render the settings view.
  - Route: `POST /dashboard/forms/:formUuid/settings` (or `/dashboard/forms/:formUuid/update-name`) to handle the update.
    - Must verify form ownership.
    - Update the form name in the database.
    - Redirect back to form settings page or main dashboard with a success message.

* **Submission Deletion (User-scoped):**
  - UI: "Delete" button per submission in `views/partials/_submissions_view.ejs` (with `confirm()` dialog).
    - Action: `POST /dashboard/submissions/delete/:submissionId`
  - Backend (in `src/routes/dashboard.js`):
    - Implemented `POST /dashboard/submissions/delete/:submissionId`:
      - Verifies the `req.user.id` owns the form to which the submission belongs.
      - Deletes the specific submission.
      - Redirects back to the form's submissions view (`/dashboard/submissions/:formUuid`) with message.

[X] You have is_archived. Solidify this. Users should be able to archive/unarchive.
[X] True delete should be a confirmed, rare operation.
[X] Implement user-scoped submission deletion.

## Task 2.2: Per-Form Configuration by User

- Mindset Shift: Empower users to customize their form behavior.

### Subtask 2.2.1: Database Schema Updates for forms Table:

- Objective: Add new fields to the `forms` table to support per-form email notification settings.
- Review existing fields (`thank_you_url`, `thank_you_message`, `ntfy_enabled`, `allowed_domains`) - these are good as per plan.
- **New fields to add:**
  - `email_notifications_enabled` (BOOLEAN, DEFAULT FALSE, NOT NULL)
  - `notification_email_address` (VARCHAR(255), NULL) - This will store an override email address. If NULL, the user's primary email will be used.

[X] Review existing fields (thank_you_url, thank_you_message, ntfy_enabled, allowed_domains). These are good.
[X] Add email_notifications_enabled (boolean). (Added to `init.sql`)
[X] Add notification_email_address (string, defaults to user's email, but allow override). (Added to `init.sql`)

### Subtask 2.2.2: UI for Form Settings Page:

- Objective: Enhance the form settings page to allow users to configure these new email notification options.
- The existing form settings page is `dashboard.ejs` with `view = 'form_settings'` (created in Subtask 2.1.5).
- **UI Elements to add to this page:**
  - **Email Notifications Section:**
    - Checkbox/Toggle: "Enable Email Notifications for new submissions"
      - Controls `email_notifications_enabled`.
    - Input Field (text, email type): "Notification Email Address"
      - Controls `notification_email_address`.
      - Should be pre-filled with the user's primary email if `notification_email_address` is NULL/empty in the DB.
      - Label should indicate that if left blank, notifications will go to the account email.
- The `GET /dashboard/forms/:formUuid/settings` route will need to fetch these new fields.
- The form on this page will need to be updated to submit these new fields. The POST route will likely be `/dashboard/forms/:formUuid/settings/update-notifications` or similar, or a general update to the existing `/dashboard/forms/:formUuid/settings/update-name` to become a general settings update route.

[X] Create a dedicated page/modal for each form's settings. (Using existing settings section in `dashboard.ejs`)
[X] Allow users to edit: Name, Email Notification toggle, Notification Email Address. (Thank You URL, Thank You Message, Allowed Domains are placeholders for now as per 2.1.5).
_ UI elements added to `dashboard.ejs` in the `form_settings` view.
_ `GET /dashboard/forms/:formUuid/settings` in `src/routes/dashboard.js` updated to fetch and pass these settings. \* `POST /dashboard/forms/:formUuid/settings/update-notifications` in `src/routes/dashboard.js` created to save these settings.

### Subtask 2.2.3: Backend to Save and Apply Settings:

- Objective: Ensure the backend API endpoints correctly save and the submission logic uses these settings.
- API endpoints to update settings for a specific form (owned by user):
  - `POST .../update-name` (Done in 2.1.5)
  - `POST .../update-notifications` (Done in 2.2.2)
  - Future: endpoints for Thank You URL, Message, Allowed Domains.
- Logic in `/submit/:formUuid` to use these form-specific settings:
  - When a form is submitted to `/submit/:formUuid`:
    - Fetch the form's settings from the DB, including `email_notifications_enabled` and `notification_email_address`.
    - This logic is now implemented in `src/routes/public.js` as part of Task 2.3.2 integration.

[X] API endpoints to update these settings for a specific form (owned by user). (Name and Email Notification settings covered so far)
[X] Logic in /submit/:formUuid to use these form-specific settings. (Addressed as part of 2.3.2)

## Task 2.3: Email Notifications for Submissions (Core Feature)

- Mindset Shift: Ntfy is cool for you. Users expect email.

### Subtask 2.3.1: Integrate Transactional Email Service:

- Objective: Set up a third-party email service to send submission notifications.
- **Action for you (USER):**
  - Choose a transactional email service (e.g., SendGrid, Mailgun, AWS SES). Many offer free tiers.
  - Sign up for the service and obtain an API Key.
  - Securely store this API Key as an environment variable in your `.env` file.
    - For example, if you choose SendGrid, you might use `SENDGRID_API_KEY=your_actual_api_key`.
    - Also, note the sender email address you configure with the service (e.g., `EMAIL_FROM_ADDRESS=notifications@yourdomain.com`).
- Once you have these, let me know which service you've chosen so I can help with installing the correct SDK and writing the integration code.
  - User selected: Resend
  - API Key ENV Var: `RESEND_API_KEY`
  - From Email ENV Var: `EMAIL_FROM_ADDRESS`

[X] Sign up for SendGrid, Mailgun, AWS SES (free tiers available). (User selected Resend)
[X] Install their SDK. (npm install resend done)
[X] Store API key securely (env vars). (User confirmed `RESEND_API_KEY` and `EMAIL_FROM_ADDRESS` are set up)

### Subtask 2.3.2: Email Sending Logic:

- Objective: Create a reusable service/function to handle the sending of submission notification emails.
- This service will use the Resend SDK and the configured API key.
- **Create a new service file:** `src/services/emailService.js`
  - It should export a function, e.g., `sendSubmissionNotification(form, submissionData, userEmail)`.
    - `form`: An object containing form details (`name`, `email_notifications_enabled`, `notification_email_address`).
    - `submissionData`: The actual data submitted to the form.
    - `userEmail`: The email of the user who owns the form (to be used if `form.notification_email_address` is not set).
  - Inside the function:
    - Check if `form.email_notifications_enabled` is true.
    - Determine the recipient: `form.notification_email_address` or `userEmail`.
    - Construct the email subject and body (using a basic template for now - Subtask 2.3.3).
    - Use the Resend SDK to send the email.
    - Include error handling (Subtask 2.3.4).

[X] Create a service/function sendSubmissionNotification(form, submissionData, userEmail) - (`src/services/emailService.js` created with this function).
[X] If email_notifications_enabled for the form, send an email to notification_email_address (or user's email). - (Logic implemented in `emailService.js` and integrated into `/submit/:formUuid` route in `src/routes/public.js`).

### Subtask 2.3.3: Basic Email Template:

- Objective: Define a simple, clear email template for notifications.
- The current `createEmailHtmlBody` function in `src/services/emailService.js` provides a very basic HTML template:
  - Subject: "New Submission for [Form Name]"
  - Body: Lists submitted data (key-value pairs).
- This fulfills the MVP requirement.

[X] Simple, clear email: "New Submission for [Form Name]", list submitted data. (Implemented in `emailService.js`)

### Subtask 2.3.4: Error Handling for Email Sending:

- Objective: Ensure email sending failures don't break the submission flow and are logged.
- In `src/services/emailService.js`, within `sendSubmissionNotification`:
  - Errors from `resend.emails.send()` are caught and logged.
  - The function does not throw an error that would halt the caller, allowing the submission to be considered successful even if the email fails.
- In `src/routes/public.js` (`/submit/:formUuid` route):
  - The call to `sendSubmissionNotification` is followed by `.catch()` to log any unexpected errors from the email sending promise itself, ensuring the main response to the user is not blocked.

[X] Log errors if email fails to send; don't let it break the submission flow. (Implemented in `emailService.js` and `public.js` route)

## Task 2.4: Enhanced Spam Protection (Beyond Basic Honeypot)

- Mindset Shift: Your honeypot is step 1. Real services need more.

### Subtask 2.4.1: Integrate CAPTCHA (e.g., Google reCAPTCHA):

- Objective: Add server-side CAPTCHA validation to the form submission process.
- We'll use Google reCAPTCHA v2 ("I'm not a robot" checkbox) for this MVP.
- **Action for you (USER):**
  - Go to the [Google reCAPTCHA admin console](https://www.google.com/recaptcha/admin/create).
  - Register your site: Choose reCAPTCHA v2, then "I'm not a robot" Checkbox.
  - Add your domain(s) (e.g., `localhost` for development, and your production domain).
  - Accept the terms of service.
  - You will receive a **Site Key** and a **Secret Key**.
  - Store these securely in your `.env` file:
    - `RECAPTCHA_V2_SITE_KEY=your_site_key`
    - `RECAPTCHA_V2_SECRET_KEY=your_secret_key`
- Let me know once you have these keys set up in your `.env` file.

- **Frontend Changes (Illustrative - User will implement on their actual forms):**
  - User needs to include the reCAPTCHA API script in their HTML form page: `<script src="https://www.google.com/recaptcha/api.js" async defer></script>`
  - User needs to add the reCAPTCHA widget div where the checkbox should appear: `<div class="g-recaptcha" data-sitekey="YOUR_RECAPTCHA_V2_SITE_KEY"></div>` (replacing with the actual site key, possibly passed from server or configured client-side if site key is public).
- **Backend Changes (`/submit/:formUuid` route in `src/routes/public.js`):**
  - When a submission is received, it should include a `g-recaptcha-response` field from the reCAPTCHA widget.
  - Create a new middleware or a helper function `verifyRecaptcha(recaptchaResponse, clientIp)`.
    - This function will make a POST request to Google's verification URL: `https://www.google.com/recaptcha/api/siteverify`.
    - Parameters: `secret` (your `RECAPTCHA_V2_SECRET_KEY`), `response` (the `g-recaptcha-response` value), `remoteip` (optional, user's IP).
    - The response from Google will be JSON indicating success or failure.
  - In the `/submit` route, call this verification function. If verification fails, reject the submission with an appropriate error.

[X] Sign up for reCAPTCHA (v2 "I'm not a robot" or v3 invisible). Get site/secret keys. (User action) - _User confirmed keys are in .env_
[ ] Frontend: Add reCAPTCHA widget/JS to user's HTML form example. (User responsibility for their forms)
[X] Backend: /submit/:formUuid endpoint must verify reCAPTCHA token with Google. (_Already implemented in `src/routes/public.js` using `src/utils/recaptchaHelper.js`_)

### Subtask 2.4.2: User Configuration for Spam Protection:

- [x] Database Schema: Add `recaptcha_enabled` (BOOLEAN, DEFAULT FALSE) to `forms` table. (_Done in `init.sql`_)
- [x] UI: Added reCAPTCHA toggle to Form Settings page (`dashboard.ejs`) and consolidated settings form to POST to `/dashboard/forms/:formUuid/settings/update`. (_Done_)
- [x] Backend:
  - [x] `GET /dashboard/forms/:formUuid/settings` fetches and passes `recaptcha_enabled`. (_Done_)
  - [x] Consolidated `POST /dashboard/forms/:formUuid/settings/update` saves `recaptcha_enabled` and other settings (formName, emailNotificationsEnabled, notificationEmailAddress). (_Done_)
  - [x] `/submit/:formUuid` in `public.js` now checks form's `recaptcha_enabled` flag: if true, token is required & verified; if false, check is skipped. (_Done_)
- [x] Allow users to enable/disable reCAPTCHA for their forms (and input their own site key if they want, or use a global one you provide). - _Implemented using global keys for MVP._

- Subtask 2.4.3: (Future Consideration) Akismet / Content Analysis.

## Task 2.5: Basic API for Users to Access Their Data

- Mindset Shift: Power users and integrations need an API.

### Subtask 2.5.1: API Key Generation & Management:

- Objective: Allow users to generate/revoke API keys from their dashboard.
- **Action for you (USER):**
  - Choose a RESTful API framework (e.g., Express, Fastify).
  - Implement the API endpoints to allow users to access their data.
  - Ensure the API is secure and uses authentication.
- Let me know once you have the API implemented and tested.

[X] Database Schema: Create `api_keys` table (user*id, key_name, api_key_identifier, hashed_api_key_secret, etc.). (\_Done in `init.sql` with refined structure*)
[X] Helper Utilities: Created `src/utils/apiKeyHelper.js` with `generateApiKeyParts`, `hashApiKeySecret`, `compareApiKeySecret`. (_Done_)
[X] Backend Routes: Added `GET /dashboard/api-keys` (list), `POST /dashboard/api-keys/generate` (create), `POST /dashboard/api-keys/:apiKeyUuid/revoke` (delete) to `src/routes/dashboard.js`. (_Done_)
[X] UI in Dashboard: Added "API Keys" section to `dashboard.ejs` for generating, listing (name, identifier, created/last*used), and revoking keys. Displays newly generated key once via session. (\_Done*)
[X] Allow users to generate/revoke API keys from their dashboard. (_Done_)
[X] Store hashed API keys in DB, associated with user. (_Done via backend routes and helpers_)

### Subtask 2.5.2: Secure API Endpoints:

- Objective: Ensure the API is secure and uses authentication.
- **Action for you (USER):**
  - Choose a RESTful API framework (e.g., Express, Fastify).
  - Implement the API endpoints to allow users to access their data.
  - Ensure the API is secure and uses authentication.
- Let me know once you have the API implemented and tested.

[X] Created `src/middleware/apiAuthMiddleware.js` for Bearer token authentication (checks signature, expiry, active user, updates last*used). (\_Done*)
[X] Created `src/routes/api_v1.js` and mounted it at `/api/v1` in `server.js`. (_Done_)
[X] Added `GET /api/v1/forms` (list user's forms) and `GET /api/v1/forms/:formUuid/submissions` (list form submissions, paginated), both protected by the API auth middleware. (_Done_)
[X] Create new API routes (e.g., /api/v1/forms, /api/v1/forms/:uuid/submissions). (_Covered by above point_)
[X] Authenticate using API keys (e.g., Bearer token). (_Done_)

### Subtask 2.5.3: Basic API Documentation:

- Objective: Provide basic documentation for the API.
- **Action for you (USER):**
  - Choose a documentation format (e.g., Swagger, Postman, Markdown).
  - Implement the documentation for the API endpoints.
- Let me know once you have the API documentation implemented.

[ ] Simple Markdown file explaining authentication and available endpoints.