Tips from open-source: Use “Set” to remove duplicates from an array.

This tip is picked from Next.js source. In this article, you will learn how to remove duplicates from an array using Set in Javascript.

From this comment above in the picture, I noticed you could remove duplicates from an array using Set.

You can execute the code below as an example to see this in action:

Learn the best practices used in open source

// Suppose exportPathMap is an object containing paths
const exportPathMap = {
  "/home": true,
  "/about": true,
  "/contact": true,
  "/home": true // Duplicate path
};

// Sample normalizePagePath function
function normalizePagePath(path) {
  return path.toLowerCase(); // Convert path to lowercase
}

// Sample denormalizePagePath function
function denormalizePagePath(path) {
  return path.toUpperCase(); // Convert path to uppercase
}

// execute the code
const exportPaths = [
  ...new Set(
    Object.keys(exportPathMap).map((path) =>
      denormalizePagePath(normalizePagePath(path))
    )
  ),
];

console.log(exportPaths); // Output: ["/HOME", "/ABOUT", "/CONTACT"]

denormalizePagePath and normalizePagePath are functions from Next.js source code. I have picked a simpler functions to demonstrate this example.

It is simple:

1. Define a Set with an array that can contain duplicates

2. Spread the Set into a new array as shown above.

denormalizePagePath(normalizePagePath(path))

I asked myself “What? why would you normalize and then denormalize?” These functions code is provided below. You first normalize to ensure paths meet certain requirements, in this case, appending /index to the path and then denormalize it by removing /index. This is to ensure consistency in the page paths. Take away here is, if you are unsure about certain variable values, normalize them aka set a standard to make it consistent and then denormalize.

normalize-page-path.ts

// source: https://github.com/vercel/next.js/blob/canary/packages/next/src/shared/lib/page-path/normalize-page-path.ts#L14
import { ensureLeadingSlash } from './ensure-leading-slash'
import { isDynamicRoute } from '../router/utils'
import { NormalizeError } from '../utils'

/**
 * Takes a page and transforms it into its file counterpart ensuring that the
 * output is normalized. Note this function is not idempotent because a page
 * `/index` can be referencing `/index/index.js` and `/index/index` could be
 * referencing `/index/index/index.js`. Examples:
 *  - `/` -> `/index`
 *  - `/index/foo` -> `/index/index/foo`
 *  - `/index` -> `/index/index`
 */
export function normalizePagePath(page: string): string {
  const normalized =
    /^\/index(\/|$)/.test(page) && !isDynamicRoute(page)
      ? `/index${page}`
      : page === '/'
      ? '/index'
      : ensureLeadingSlash(page)

  if (process.env.NEXT_RUNTIME !== 'edge') {
    const { posix } = require('path')
    const resolvedPage = posix.normalize(normalized)
    if (resolvedPage !== normalized) {
      throw new NormalizeError(
        `Requested and resolved page mismatch: ${normalized} ${resolvedPage}`
      )
    }
  }

  return normalized
}

denormalize-page-path:

import { isDynamicRoute } from '../router/utils'
import { normalizePathSep } from './normalize-path-sep'

/**
 * Performs the opposite transformation of `normalizePagePath`. Note that
 * this function is not idempotent either in cases where there are multiple
 * leading `/index` for the page. Examples:
 *  - `/index` -> `/`
 *  - `/index/foo` -> `/foo`
 *  - `/index/index` -> `/index`
 */
export function denormalizePagePath(page: string) {
  let _page = normalizePathSep(page)
  return _page.startsWith('/index/') && !isDynamicRoute(_page)
    ? _page.slice(6)
    : _page !== '/index'
    ? _page
    : '/'
}

About me:

Hey, my name is Ramu Narasinga. I study large open-source projects and create content about their codebase architecture and best practices, sharing it through articles, videos.

I am open to work on interesting projects. Send me an email at ramu.narasinga@gmail.com

My Github —  https://github.com/ramu-narasinga

My website —  https://ramunarasinga.com

My Youtube channel —  https://www.youtube.com/@ramu-narasinga

Learning platform —  https://thinkthroo.com

Codebase Architecture —  https://app.thinkthroo.com/architecture

Best practices —  https://app.thinkthroo.com/best-practices

Production-grade projects —  https://app.thinkthroo.com/production-grade-projects

Conclusion:

I changed my article title to “Tips from open-source” from “Lessons from open-source” as these are short concised tips extracted from opensource code to improve your coding abilities.

You can use Set to remove duplicates from an array and then convert this Set back to an array using spread operator. I also saw function calling another function as its parameter — denormalizePagePath(normalizePagePath(path)).

I have written functions using this style before to improve readability. Condensed functions => less lines of spaghetti code to deal with => improved readability.