MongoDB Realm

We use MongoDB Realm as serverless functions provider, in addition to:

1. NestJS REST API serverless functions, callable directly or from Hasura
2. NestJS GraphQL API serverless functions, callable directly or from Hasura
3. Webiny applications

The primary advantages of MongoDB Realm are: rapid experimentation & deployment using browser, built-in authentication & authorization model, custom JWT provider support, and automatic GraphQL queries & mutations (but not subscriptions).

urql Realm authentication & refreshing token lifecycle using authExchange

Reference: https://formidable.com/open-source/urql/docs/advanced/authentication/

Below is the automatic token refreshing logic for Realm access token only. For FusionAuth’s token, it is assumed that we’ll use either @fusionauth/typescript-client or Axios, so it is handled by a different mechanism.

import { createClient, dedupExchange, cacheExchange, fetchExchange, Provider } from 'urql';
import { makeOperation } from '@urql/core';
import { authExchange } from '@urql/exchange-auth';
import { processSignOut } from './services/login';

// ...

const client = createClient({
  url: appConfig.REALM_GRAPHQL_URL,
  exchanges: [
    dedupExchange,
    cacheExchange,
    // Reference: https://formidable.com/open-source/urql/docs/advanced/authentication/
    authExchange<{realmAccessToken: string, realmRefreshToken: string}>({
      addAuthToOperation: ({
        authState,
        operation,
      }) => {
        // the token isn't in the auth state, return the operation without changes
        if (!authState || !authState.realmAccessToken) {
          return operation;
        }

        // fetchOptions can be a function (See Client API) but you can simplify this based on usage
        const fetchOptions =
          typeof operation.context.fetchOptions === 'function'
            ? operation.context.fetchOptions()
            : operation.context.fetchOptions || {};

        return makeOperation(
          operation.kind,
          operation,
          {
            ...operation.context,
            fetchOptions: {
              ...fetchOptions,
              headers: {
                ...fetchOptions.headers,
                "Authorization": `Bearer ${authState.realmAccessToken}`,
              },
            },
          },
        );
      },
      willAuthError: ({ authState }) => {
        if (!authState) return true;
        // e.g. check for expiration, existence of auth etc
        return false;
      },
      didAuthError: ({ error }) => {
        // check if the error was an auth error (this can be implemented in various ways, e.g. 401 or a special error code)
        return error.response?.status === 401 || error.graphQLErrors.some(
          e => e.extensions?.code === 'FORBIDDEN',
        );
      },
      getAuth: async ({ authState, mutate }) => {
        // for initial launch, fetch the auth state from storage (local storage, async storage etc)
        if (!authState) {
          if (app.currentUser?.accessToken && app.currentUser?.refreshToken) {
            return { realmAccessToken: app.currentUser.accessToken,
              realmRefreshToken: app.currentUser.refreshToken };
          }
          // const { token, refreshToken } = readAuthFromLocalStorage();
          // if (token && refreshToken) {
          //   return { token, refreshToken };
          // }
          return null;
        }

        /**
         * the following code gets executed when an auth error has occurred
         * we should refresh the token if possible and return a new auth state
         * If refresh fails, we should log out
         **/

        // if your refresh logic is in graphQL, you must use this mutate function to call it
        // if your refresh logic is a separate RESTful endpoint, use fetch or similar
        if (app.currentUser) {
          try {
            console.debug('Refreshing Realm token for', app.currentUser.id, '...');
            await app.currentUser?.refreshAccessToken();

            // save the new tokens in storage for next restart
            writeRealmAuthToLocalStorage({realmUserId: app.currentUser.id,
              realmAccessToken: app.currentUser.accessToken!,
              realmRefreshToken: app.currentUser.refreshToken!});
  
            // return the new tokens
            return {
              realmAccessToken: app.currentUser.accessToken!,
              realmRefreshToken: app.currentUser.refreshToken!,
            };
          } catch (err) {
            console.error('Cannot refresh Realm token for', app.currentUser.id, ', will sign out');
          }
        }

        // otherwise, if refresh fails, log clear storage and log out
        await processSignOut();

        return null;
      },
    }),
    fetchExchange,
  ],
});

The application-side functions used are:

export function writeRealmAuthToLocalStorage({realmUserId, realmAccessToken, realmRefreshToken}:
  {realmUserId: string, realmAccessToken: string, realmRefreshToken: string}) {
  console.debug('Writing to localStorage: realmUserId, realmAccessToken, realmRefreshToken');
  localStorage.setItem('realmUserId', realmUserId);
  localStorage.setItem('realmAccessToken', realmAccessToken);
  localStorage.setItem('realmRefreshToken', realmRefreshToken);
}

export function removeAuthFromLocalStorage() {
  console.debug('Removing auth from localStorage: token, refreshToken, currentUser, realmUserId, realmAccessToken, realmRefreshToken');
  localStorage.removeItem('token');
  localStorage.removeItem('refreshToken');
  localStorage.removeItem('currentUser');
  localStorage.removeItem('realmUserId');
  localStorage.removeItem('realmAccessToken');
  localStorage.removeItem('realmRefreshToken');
}

export async function processSignOut() {
  if (app.currentUser) {
    console.info('Signing out', app.currentUser.id, 'from Realm...');
    try {
      await app.currentUser.logOut();
    } catch (err) {
      console.warn('Did not log out Realm user (but ignored):', err);
    }
  }
  const refreshToken = localStorage.getItem('refreshToken');
  if (refreshToken) {
    const fusionauthClient = new FusionAuthClient(appConfig.FUSIONAUTH_CLIENT_KEY, appConfig.FUSIONAUTH_URL,
      appConfig.FUSIONAUTH_TENANT_ID);
    console.info('Signing out...');
    const res = await fusionauthClient.logout(false, refreshToken);
    console.debug('Signed out from ', appConfig.FUSIONAUTH_URL, ':', res);
  } else {
    console.warn('processSignOut() called but refreshToken is null');
  }
  removeAuthFromLocalStorage();
}

GraphQL

Troubleshooting: role ROLE in DATABASE does not have update permission for document with id ID: could not validate document …..

Unforunately

Error: [GraphQL] reason=”role \”owner\” in \”lovia_staging.user\” does not have update permission for document with _id: 466: could not validate document: \n\temailBounceTime: Invalid type. Expected: undefined, given: null\n\thijabKind: Invalid type. Expected: undefined, given: null\n\timpressionCount: Invalid type. Expected: type: undefined, bsonType: double, given: [integer int long number mixed]\n\tinactiveReason: Invalid type. Expected: undefined, given: null\n\tinactiveTime: Invalid type. Expected: undefined, given: null\n\tmobileBounceTime: Invalid type. Expected: undefined, given: null\n\tpremiumAmount: Invalid type. Expected: undefined, given: null\n\tsuspendReason: Invalid type. Expected: undefined, given: null\n\tsuspendTime: Invalid type. Expected: undefined, given: null\n\tviewCount: Invalid type. Expected: type: undefined, bsonType: double, given: [integer int long number mixed]”; code=”SchemaValidationFailedWrite”; untrusted=”update not permitted”; details=map[]

Error: [GraphQL] reason=”role \”owner\” in \”lovia_staging.user\” does not have update permission for document with _id: 466: could not validate document: \n\temailBounceTime: Invalid type. Expected: undefined, given: null\n\thijabKind: Invalid type. Expected: undefined, given: null\n\timpressionCount: Invalid type. Expected: type: undefined, bsonType: double, given: [integer int long number mixed]\n\tinactiveReason: Invalid type. Expected: undefined, given: null\n\tinactiveTime: Invalid type. Expected: undefined, given: null\n\tmobileBounceTime: Invalid type. Expected: undefined, given: null\n\tpremiumAmount: Invalid type. Expected: undefined, given: null\n\tsuspendReason: Invalid type. Expected: undefined, given: null\n\tsuspendTime: Invalid type. Expected: undefined, given: null\n\tviewCount: Invalid type. Expected: type: undefined, bsonType: double, given: [integer int long number mixed]”; code=”SchemaValidationFailedWrite”; untrusted=”update not permitted”; details=map[]

Actual Cause: The JSON Schema only allows concrete type (e.g. string) and undefined (meaning missing value). Therefore, null is forbidden.

Bad Workaround: As I described in this MongoDB Realm thread, you can change MongoDB Realm App > GraphQL > Settings > Validation Action to only Warn.. However that will only work for GraphQL, and will not work for regular functions. So the correct way is to fix the data. See thread.

Worse Workaround: If you try to modify the JSON Schema as follows, GraphQL will not work at all.

"bsonType": [ "string", "null" ]

Use the following script to fix your current schema:

// let schema = ...
for (const key of Object.keys(schema.properties)) {
  if (typeof schema.properties[key].bsonType === 'string') {
    schema.properties[key].bsonType = [ schema.properties[key].bsonType, "null" ];
  }
}
console.log(JSON.stringify(schema, null, 2));

Realm GraphQL does not support properties with multiple bsonType’s.

The proper solution would be not to use null fields at all, and set them to undefined (unset). You can use the following script as a start. IMPORTANT: Doing the updateMany will take a long time (maybe 1 hour or more), because table scans are necessary for each field, especially when not indexed.

// let schema = ...
for (const key of Object.keys(schema.properties)) {
  console.log(`db.user.updateMany({ ${key}: {$type: 10} }, {$unset: { ${key}: 1} });`);
}

// alternatively if you just want to update one:
for (const key of Object.keys(schema.properties)) {
  console.log(`db.user.updateMany({ _id: 466, ${key}: {$type: 10} }, {$unset: { ${key}: 1} });`);
}

Note: For Lovia’s user collection, make sure impressionCount and viewCount are “bsonType”: “int” and not “double”.

If you need to convert a field to another type, $toDouble (aggregation) and other aggregation operators may be useful (using MongoDB’s recent update using aggregation feature).