formies/notes.md

Task 2.1: User Dashboard & Form Management UI (Replacing current "admin")

• Mindset Shift: This is no longer an admin panel. It's the user's control center.

Subtask 2.1.1: Design User Dashboard Layout

• Wireframe basic layout:
  • Navigation Bar:
    • Logo/App Name (e.g., "Formies")
    • My Forms (Active Link)
    • Create New Form
    • Account Settings (e.g., "Hi, [User Name]" dropdown with "Settings", "Logout")
  • Main Content Area (for "My Forms" view):
    • Header: "My Forms"
    • Button: "+ Create New Form"
    • Forms List Table:
      • Columns: Form Name, Submissions (count), Endpoint URL, Created Date, Actions
      • Actions per row: View Submissions, Settings, Archive/Delete
    • Pagination for the forms list if it becomes long.
  • Main Content Area (for "Create New Form" view - initial thought, might be a separate page/modal):
    • Header: "Create New Form"
    • Form fields: Form Name
    • Button: "Create Form"
  • Main Content Area (for "Account Settings" - placeholder for now):
    • Header: "Account Settings"
    • Placeholder content.
• Frontend Tech Decision:
  • EJS for templating, made dynamic with client-side JavaScript. This aligns with the existing structure and MVP scope. We will enhance EJS views to be more interactive.

[X] Wireframe basic layout: List forms, create form, account settings (placeholder). - Textual wireframe defined above [X] Decide on frontend tech (EJS is fine for MVP if you make it dynamic with client-side JS, or consider a simple SPA framework if comfortable). - Decision made: EJS with client-side JS

• Created views/dashboard.ejs as the main layout.
• Created views/partials/_forms_table.ejs for displaying the list of forms.

Subtask 2.1.2: "My Forms" View:

• Objective: Fetch and display forms owned by the logged-in user.
• Show key info: name, submission count, endpoint URL, created date, status (Active/Archived).
• Links/Actions: View Submissions, Settings, Archive/Unarchive, Delete.
• Frontend: views/dashboard.ejs with view = 'my_forms' and views/partials/_forms_table.ejs will handle this.
• Backend:
  • Need a new route, e.g., GET /dashboard, protected by authentication (e.g., requireAuth from authMiddleware.js).
  • This route handler will:
    • Fetch forms for req.user.id from the database.
    • Query should include name, uuid, created_at, is_archived, and submission_count.
    • Render views/dashboard.ejs passing the forms data, user object, appUrl, and view = 'my_forms'.
  • Implemented in src/routes/dashboard.js via GET /.

[X] Fetch and display forms owned by the logged-in user. [X] Show key info: name, submission count, endpoint URL, created date. [X] Links to: view submissions, edit settings, delete. (Links are present in _forms_table.ejs, functionality for all to be built out in subsequent tasks)

Subtask 2.1.3: "Create New Form" Functionality (for logged-in user):

• UI: dashboard.ejs (with view = 'create_form') provides the form input.
  • Route GET /dashboard/create-form in src/routes/dashboard.js renders this view.
• Backend: POST /dashboard/forms/create route in src/routes/dashboard.js handles form submission.
  • Associates form with req.user.id.
  • Redirects to /dashboard on success.
  • Handles errors and re-renders create form view with an error message.

[X] UI and backend logic. Associates form with req.user.id.

Subtask 2.1.4: "View Form Submissions" (Scoped & Paginated):

• Objective: Allow users to view submissions for their specific forms, with pagination.
• UI:
  • views/partials/_submissions_view.ejs created to display submissions list and pagination.
  • views/dashboard.ejs updated to include this partial when view = 'form_submissions'.
• Backend:
  • Route: GET /dashboard/submissions/:formUuid added to src/routes/dashboard.js.
  • Verifies that req.user.id owns the formUuid.
  • Fetches paginated submissions for the given formUuid.
  • Renders dashboard.ejs with view = 'form_submissions', passing submissions data, form details, and pagination info.
  • Error handling improved to render user-friendly messages within the dashboard view.

[X] UI and backend for a user to view submissions for their specific form. [X] Pagination is critical here (as you have).

Subtask 2.1.5: Form Settings UI (Basic):

• Objective: Allow users to update basic form settings, starting with the form name.
• UI:
  • A new view/section in dashboard.ejs (e.g., when view = 'form_settings').
  • This view will display a form with an input for the form name.
  • It will also be a placeholder for future settings (thank you URL, notifications).
• Backend:
  • Route: GET /dashboard/forms/:formUuid/settings to display the settings page.
    • Implemented in src/routes/dashboard.js.
    • Verifies form ownership by req.user.id.
    • Fetches current form details (name).
    • Renders the form_settings view in dashboard.ejs.
  • Route: POST /dashboard/forms/:formUuid/settings/update-name to handle the update.
    • Implemented in src/routes/dashboard.js.
    • Verifies form ownership.
    • Updates the form name in the database.
    • Redirects back to form settings page with a success/error message via query parameters.

[X] Allow users to update form name. [X] Placeholder for future settings (thank you URL, notifications) - (Placeholders added in EJS).

Subtask 2.1.6: Delete Form/Submission (with soft delete/archival consideration):

• Objective: Implement form archival (soft delete) and permanent deletion for users.

• Users should be able to archive/unarchive their forms.

• True delete should be a confirmed, rare operation.

• The is_archived field in the forms table will be used.

• Submissions deletion is already partially handled in _submissions_view.ejs via a POST to /dashboard/submissions/delete/:submissionId. We need to implement this backend route.

• Form Archival/Unarchival:

  • UI: Buttons for "Archive" / "Unarchive" are already in views/partials/_forms_table.ejs.
    • Archive action: POST /dashboard/forms/archive/:formUuid
    • Unarchive action: POST /dashboard/forms/unarchive/:formUuid
  • Backend:
    • Create these two POST routes in src/routes/dashboard.js.
    • Must verify form ownership by req.user.id.
    • Fetch current form details (name).
    • Render the settings view.
  • Route: POST /dashboard/forms/:formUuid/settings (or /dashboard/forms/:formUuid/update-name) to handle the update.
    • Must verify form ownership.
    • Update the form name in the database.
    • Redirect back to form settings page or main dashboard with a success message.

• Submission Deletion (User-scoped):
  • UI: "Delete" button per submission in views/partials/_submissions_view.ejs (with confirm() dialog).
    • Action: POST /dashboard/submissions/delete/:submissionId
  • Backend (in src/routes/dashboard.js):
    • Implemented POST /dashboard/submissions/delete/:submissionId:
      • Verifies the req.user.id owns the form to which the submission belongs.
      • Deletes the specific submission.
      • Redirects back to the form's submissions view (/dashboard/submissions/:formUuid) with message.

[X] You have is_archived. Solidify this. Users should be able to archive/unarchive. [X] True delete should be a confirmed, rare operation. [X] Implement user-scoped submission deletion.

Task 2.2: Per-Form Configuration by User

• Mindset Shift: Empower users to customize their form behavior.

Subtask 2.2.1: Database Schema Updates for forms Table:

• Objective: Add new fields to the forms table to support per-form email notification settings.
• Review existing fields (thank_you_url, thank_you_message, ntfy_enabled, allowed_domains) - these are good as per plan.
• New fields to add:
  • email_notifications_enabled (BOOLEAN, DEFAULT FALSE, NOT NULL)
  • notification_email_address (VARCHAR(255), NULL) - This will store an override email address. If NULL, the user's primary email will be used.

[X] Review existing fields (thank_you_url, thank_you_message, ntfy_enabled, allowed_domains). These are good. [X] Add email_notifications_enabled (boolean). (Added to init.sql) [X] Add notification_email_address (string, defaults to user's email, but allow override). (Added to init.sql)

Subtask 2.2.2: UI for Form Settings Page:

• Objective: Enhance the form settings page to allow users to configure these new email notification options.
• The existing form settings page is dashboard.ejs with view = 'form_settings' (created in Subtask 2.1.5).
• UI Elements to add to this page:
  • Email Notifications Section:
    • Checkbox/Toggle: "Enable Email Notifications for new submissions"
      • Controls email_notifications_enabled.
    • Input Field (text, email type): "Notification Email Address"
      • Controls notification_email_address.
      • Should be pre-filled with the user's primary email if notification_email_address is NULL/empty in the DB.
      • Label should indicate that if left blank, notifications will go to the account email.
• The GET /dashboard/forms/:formUuid/settings route will need to fetch these new fields.
• The form on this page will need to be updated to submit these new fields. The POST route will likely be /dashboard/forms/:formUuid/settings/update-notifications or similar, or a general update to the existing /dashboard/forms/:formUuid/settings/update-name to become a general settings update route.

[X] Create a dedicated page/modal for each form's settings. (Using existing settings section in dashboard.ejs) [X] Allow users to edit: Name, Email Notification toggle, Notification Email Address. (Thank You URL, Thank You Message, Allowed Domains are placeholders for now as per 2.1.5). _ UI elements added to dashboard.ejs in the form_settings view. _ GET /dashboard/forms/:formUuid/settings in src/routes/dashboard.js updated to fetch and pass these settings. * POST /dashboard/forms/:formUuid/settings/update-notifications in src/routes/dashboard.js created to save these settings.

Subtask 2.2.3: Backend to Save and Apply Settings:

• Objective: Ensure the backend API endpoints correctly save and the submission logic uses these settings.
• API endpoints to update settings for a specific form (owned by user):
  • POST .../update-name (Done in 2.1.5)
  • POST .../update-notifications (Done in 2.2.2)
  • Future: endpoints for Thank You URL, Message, Allowed Domains.
• Logic in /submit/:formUuid to use these form-specific settings:
  • When a form is submitted to /submit/:formUuid:
    • Fetch the form's settings from the DB, including email_notifications_enabled and notification_email_address.
    • This logic is now implemented in src/routes/public.js as part of Task 2.3.2 integration.

[X] API endpoints to update these settings for a specific form (owned by user). (Name and Email Notification settings covered so far) [X] Logic in /submit/:formUuid to use these form-specific settings. (Addressed as part of 2.3.2)

Task 2.3: Email Notifications for Submissions (Core Feature)

• Mindset Shift: Ntfy is cool for you. Users expect email.

Subtask 2.3.1: Integrate Transactional Email Service:

• Objective: Set up a third-party email service to send submission notifications.
• Action for you (USER):
  • Choose a transactional email service (e.g., SendGrid, Mailgun, AWS SES). Many offer free tiers.
  • Sign up for the service and obtain an API Key.
  • Securely store this API Key as an environment variable in your .env file.
    • For example, if you choose SendGrid, you might use SENDGRID_API_KEY=your_actual_api_key.
    • Also, note the sender email address you configure with the service (e.g., EMAIL_FROM_ADDRESS=notifications@yourdomain.com).
• Once you have these, let me know which service you've chosen so I can help with installing the correct SDK and writing the integration code.
  • User selected: Resend
  • API Key ENV Var: RESEND_API_KEY
  • From Email ENV Var: EMAIL_FROM_ADDRESS

[X] Sign up for SendGrid, Mailgun, AWS SES (free tiers available). (User selected Resend) [X] Install their SDK. (npm install resend done) [X] Store API key securely (env vars). (User confirmed RESEND_API_KEY and EMAIL_FROM_ADDRESS are set up)

Subtask 2.3.2: Email Sending Logic:

• Objective: Create a reusable service/function to handle the sending of submission notification emails.
• This service will use the Resend SDK and the configured API key.
• Create a new service file: src/services/emailService.js
  • It should export a function, e.g., sendSubmissionNotification(form, submissionData, userEmail).
    • form: An object containing form details (name, email_notifications_enabled, notification_email_address).
    • submissionData: The actual data submitted to the form.
    • userEmail: The email of the user who owns the form (to be used if form.notification_email_address is not set).
  • Inside the function:
    • Check if form.email_notifications_enabled is true.
    • Determine the recipient: form.notification_email_address or userEmail.
    • Construct the email subject and body (using a basic template for now - Subtask 2.3.3).
    • Use the Resend SDK to send the email.
    • Include error handling (Subtask 2.3.4).

[X] Create a service/function sendSubmissionNotification(form, submissionData, userEmail) - (src/services/emailService.js created with this function). [X] If email_notifications_enabled for the form, send an email to notification_email_address (or user's email). - (Logic implemented in emailService.js and integrated into /submit/:formUuid route in src/routes/public.js).

Subtask 2.3.3: Basic Email Template:

• Objective: Define a simple, clear email template for notifications.
• The current createEmailHtmlBody function in src/services/emailService.js provides a very basic HTML template:
  • Subject: "New Submission for [Form Name]"
  • Body: Lists submitted data (key-value pairs).
• This fulfills the MVP requirement.

[X] Simple, clear email: "New Submission for [Form Name]", list submitted data. (Implemented in emailService.js)

Subtask 2.3.4: Error Handling for Email Sending:

• Objective: Ensure email sending failures don't break the submission flow and are logged.
• In src/services/emailService.js, within sendSubmissionNotification:
  • Errors from resend.emails.send() are caught and logged.
  • The function does not throw an error that would halt the caller, allowing the submission to be considered successful even if the email fails.
• In src/routes/public.js (/submit/:formUuid route):
  • The call to sendSubmissionNotification is followed by .catch() to log any unexpected errors from the email sending promise itself, ensuring the main response to the user is not blocked.

[X] Log errors if email fails to send; don't let it break the submission flow. (Implemented in emailService.js and public.js route)

Task 2.4: Enhanced Spam Protection (Beyond Basic Honeypot)

• Mindset Shift: Your honeypot is step 1. Real services need more.

Subtask 2.4.1: Integrate CAPTCHA (e.g., Google reCAPTCHA):

• Objective: Add server-side CAPTCHA validation to the form submission process.

• We'll use Google reCAPTCHA v2 ("I'm not a robot" checkbox) for this MVP.

• Action for you (USER):

  • Go to the Google reCAPTCHA admin console.
  • Register your site: Choose reCAPTCHA v2, then "I'm not a robot" Checkbox.
  • Add your domain(s) (e.g., localhost for development, and your production domain).
  • Accept the terms of service.
  • You will receive a Site Key and a Secret Key.
  • Store these securely in your .env file:
    • RECAPTCHA_V2_SITE_KEY=your_site_key
    • RECAPTCHA_V2_SECRET_KEY=your_secret_key

• Let me know once you have these keys set up in your .env file.

• Frontend Changes (Illustrative - User will implement on their actual forms):

  • User needs to include the reCAPTCHA API script in their HTML form page: <script src="https://www.google.com/recaptcha/api.js" async defer></script>
  • User needs to add the reCAPTCHA widget div where the checkbox should appear: <div class="g-recaptcha" data-sitekey="YOUR_RECAPTCHA_V2_SITE_KEY"></div> (replacing with the actual site key, possibly passed from server or configured client-side if site key is public).

• Backend Changes (/submit/:formUuid route in src/routes/public.js):

  • When a submission is received, it should include a g-recaptcha-response field from the reCAPTCHA widget.
  • Create a new middleware or a helper function verifyRecaptcha(recaptchaResponse, clientIp).
    • This function will make a POST request to Google's verification URL: https://www.google.com/recaptcha/api/siteverify.
    • Parameters: secret (your RECAPTCHA_V2_SECRET_KEY), response (the g-recaptcha-response value), remoteip (optional, user's IP).
    • The response from Google will be JSON indicating success or failure.
  • In the /submit route, call this verification function. If verification fails, reject the submission with an appropriate error.

[X] Sign up for reCAPTCHA (v2 "I'm not a robot" or v3 invisible). Get site/secret keys. (User action) - User confirmed keys are in .env [ ] Frontend: Add reCAPTCHA widget/JS to user's HTML form example. (User responsibility for their forms) [X] Backend: /submit/:formUuid endpoint must verify reCAPTCHA token with Google. (Already implemented in src/routes/public.js using src/utils/recaptchaHelper.js)

Subtask 2.4.2: User Configuration for Spam Protection:

• Database Schema: Add recaptcha_enabled (BOOLEAN, DEFAULT FALSE) to forms table. (Done in init.sql)

• UI: Added reCAPTCHA toggle to Form Settings page (dashboard.ejs) and consolidated settings form to POST to /dashboard/forms/:formUuid/settings/update. (Done)

• Backend:

  • GET /dashboard/forms/:formUuid/settings fetches and passes recaptcha_enabled. (Done)
  • Consolidated POST /dashboard/forms/:formUuid/settings/update saves recaptcha_enabled and other settings (formName, emailNotificationsEnabled, notificationEmailAddress). (Done)
  • /submit/:formUuid in public.js now checks form's recaptcha_enabled flag: if true, token is required & verified; if false, check is skipped. (Done)

• Allow users to enable/disable reCAPTCHA for their forms (and input their own site key if they want, or use a global one you provide). - Implemented using global keys for MVP.

• Subtask 2.4.3: (Future Consideration) Akismet / Content Analysis.

Task 2.5: Basic API for Users to Access Their Data

• Mindset Shift: Power users and integrations need an API.

Subtask 2.5.1: API Key Generation & Management:

• Objective: Allow users to generate/revoke API keys from their dashboard.
• Action for you (USER):
  • Choose a RESTful API framework (e.g., Express, Fastify).
  • Implement the API endpoints to allow users to access their data.
  • Ensure the API is secure and uses authentication.
• Let me know once you have the API implemented and tested.

[X] Database Schema: Create api_keys table (userid, key_name, api_key_identifier, hashed_api_key_secret, etc.). (_Done in init.sql with refined structure) [X] Helper Utilities: Created src/utils/apiKeyHelper.js with generateApiKeyParts, hashApiKeySecret, compareApiKeySecret. (Done) [X] Backend Routes: Added GET /dashboard/api-keys (list), POST /dashboard/api-keys/generate (create), POST /dashboard/api-keys/:apiKeyUuid/revoke (delete) to src/routes/dashboard.js. (Done) [X] UI in Dashboard: Added "API Keys" section to dashboard.ejs for generating, listing (name, identifier, created/lastused), and revoking keys. Displays newly generated key once via session. (_Done) [X] Allow users to generate/revoke API keys from their dashboard. (Done) [X] Store hashed API keys in DB, associated with user. (Done via backend routes and helpers)

Subtask 2.5.2: Secure API Endpoints:

• Objective: Ensure the API is secure and uses authentication.
• Action for you (USER):
  • Choose a RESTful API framework (e.g., Express, Fastify).
  • Implement the API endpoints to allow users to access their data.
  • Ensure the API is secure and uses authentication.
• Let me know once you have the API implemented and tested.

[X] Created src/middleware/apiAuthMiddleware.js for Bearer token authentication (checks signature, expiry, active user, updates lastused). (_Done) [X] Created src/routes/api_v1.js and mounted it at /api/v1 in server.js. (Done) [X] Added GET /api/v1/forms (list user's forms) and GET /api/v1/forms/:formUuid/submissions (list form submissions, paginated), both protected by the API auth middleware. (Done) [X] Create new API routes (e.g., /api/v1/forms, /api/v1/forms/:uuid/submissions). (Covered by above point) [X] Authenticate using API keys (e.g., Bearer token). (Done)

Subtask 2.5.3: Basic API Documentation:

• Objective: Provide basic documentation for the API.
• Action for you (USER):
  • Choose a documentation format (e.g., Swagger, Postman, Markdown).
  • Implement the documentation for the API endpoints.
• Let me know once you have the API documentation implemented.

[ ] Simple Markdown file explaining authentication and available endpoints.