Simple, extensible split testing in Svelte Kit

I do a quick dive into a simple but effective method of split testing landing pages using Svelte Kit. Concepts covered in this post are great for people who are new to Svelte Kit.

Oct 02 • 10 Minute Read

Svelte • Server Side • Beginner

Simple, extensible split testing in Svelte Kit

Currently, I’m working on building a project for an online course that helps breastfeeding mothers increase their milk supply. I’ve chosen Svelte Kit for the front end, which includes a website, a Udemy-esque members area, and some landing pages for marketing. I can feel marketers cringing at this as they wonder why I don’t just use Clickfunnels , Wordpress , or Kajaba . Well, the reality is, at this point, I feel more freedom and comfort building things in a coding language than I do in some WYSIWYG editor. I may write a post on this, but for now, it suffices to say I’d rather trade some time for the freedom to build anything I want.

Along those lines, one of the requirements for the site was the ability to run some basic split testing on the landing pages. Initially, we’ll be running experiments on sales page copy and pricing. I know there is a lot of dogma in data science these days that dictates the best way of running tests, collecting and analyzing data, and choosing the best variants – but let's focus on the actual approach I took in Svelte Kit.

The implementation

High-level, I made use of Svelte Kit’s group feature by grouping all of the landing pages under a single server-side layout load function that handles split test variant selection and caching. If that means something to you, you can go straight to the repo here ; otherwise, keep reading.

Now, I’d like to start by reasoning about how the split test data might look. I know that we will be needing to run tests on page elements like colors and copy – but I’ll also be testing higher-level items like what pages constitute a particular funnel. For instance, I might decide to have an upsell page in between the sales page and the checkout page:

Instead of creating two different sales pages with two different call-to-action locations (i.e., one CTA to the checkout page and the other to the upsell page), it would be better to create some type of funnel variant object that dictates where each page's CTA should link to. Then we can reference the specific CTA data within each +page.svelte and programmatically display the proper link and page variants. Let’s model this data in a file. Starting with a fresh Svelte Kit project, we create a file 'src/lib/funnels.ts,' then model our data and export some dummy data:

src/lib/funnels.ts

export type Cta = {
  innerText: string
  href: string
}

export type FunnelElement = {
  varient: 0 | 1
  cta: Cta
}

export type PathName = string

export type Funnel = {
  id: string
  data: {
    [key: PathName]: FunnelElement
  }
}

export const a: Funnel = {
  id: 'a',
  data: {
    '/salespage': {
      varient: 0,
      cta: {
        innerText: 'I Want It',
        href: '/checkout'
      }
    },
    '/checkout': {
      varient: 1,
      cta: {
        innerText: 'Purchase',
        href: '/'
      }
    },
  }
}

export const b: Funnel = {
  id: 'b',
  data: {
    '/salespage': {
      varient: 1,
      cta: {
        innerText: 'I Want In',
        href: '/upsell'
      }
    },
    '/upsell': {
      varient: 0,
      cta: {
        innerText: 'I Want The Upsell',
        href: '/checkout'
      }
    },
  }
}

export type Funnels = {
  [key: string]: Funnel
}

export const funnels: Funnels = {
  a,
  b,
}

Let’s talk about the types at the top. I created the CTA (call to action) type that will hold the data that dictates a CTA button’s copy and link location. The ‘FunnelElement’ type will hold data that will tell the client what variant to render and the CTA copy/location. Finally, the 'Funnel' type maps a set of page path names to their respective funnel element data and contains a unique funnel ID that we can reference when split testing. For example, if we take a look at funnel ‘a,’ we know that the page route ‘/salespage’ should render variant ‘0’ and the call to action 'I Want It' linking to ‘/checkout.’

Now we need a ‘SplitTest’ data structure that will map probabilities to each element in an arbitrary set of funnels. We can then reference those probabilities on the server to assign a funnel variant to the client. We can create a file “src/lib/splits.ts,” model the split test data, and create a demo split test:

src/lib/splits.ts

import type { Funnel } from "./funnel"
import { funnels } from './funnel'

export type Varient = {
  funnel: Funnel
  probability: number
}

export type SplitTest = {
  id: string
  varients: Varient[]
}

type SplitTestId = string

type Splits = Record<SplitTestId, SplitTest>

export const splits: Splits = {
  a: {
    id: 'a',
    varients: [
      {
        funnel: funnels.a,
        probability: 0.5
      },
      {
        funnel: funnels.b,
        probability: 0.5
      }
    ]
  }
}

Now that we have our data modeled, we can create the file and directory '(marketing)/+layout.server.ts' in 'src/routes.' We can start by instantiating a server-side load function and creating an object 'EntryPoints' that dictates which split test is used based on the client's entry point:

src/routes/(marketing)/+layout.server.ts

import type { LayoutServerLoad } from './$types';
import { splits } from '$lib/splits';
import type { PathName } from '$lib/funnel';
import type { SplitTest } from '$lib/splits';

type EntryPoints = Record<PathName, SplitTest>
const entryPoints: EntryPoints = {
  '/salespage': splits.a,
  '/checkout': splits.a
}

export const load: LayoutServerLoad = async ({ cookies, url }) => {}

Within the load function, we will remove any trailing slashes on the pathname and match the pathname to our set of entry points:

src/routes/(marketing)/+layout.server.ts

import type { LayoutServerLoad } from './$types';
import { splits } from '$lib/splits';
import type { PathName } from '$lib/funnel';
import type { SplitTest } from '$lib/splits';

type EntryPoints = Record<PathName, SplitTest>
const entryPoints: EntryPoints = {
  '/salespage': splits.a,
  '/checkout': splits.a
}

export const load: LayoutServerLoad = async ({ cookies, url }) => {
  const pathname = url.pathname.replace(/\/+$/, '')
  let splitTest = entryPoints[pathname]
  if (!splitTest) {
    //no split test for this page as an entry point
  }
  //split test logic here
}

Then we can write a function in 'splits.ts' that takes a SplitTest and chooses a variant based on the probabilities:

src/lib/splits.ts

export function getVarient(v: Varient[]): Funnel {
  const rando = Math.random()
  let collectiveProb = 0

  for (const varient of v) {
    collectiveProb += varient.probability
    if (rando <= collectiveProb) {
      return varient.funnel
    }
  }

  throw new Error(`Please make the sum of probabilities equal to 1 for splits: ${JSON.stringify(v)}`)
}

And import it for use in our load function to return a funnel.

src/routes/(marketing)/+layout.server.ts

import type { LayoutServerLoad } from './$types';
import { splits, getVarient } from '$lib/splits';
import type { PathName } from '$lib/funnel';
import type { SplitTest } from '$lib/splits';

type EntryPoints = Record<PathName, SplitTest>
const entryPoints: EntryPoints = {
  '/salespage': splits.a,
  '/checkout': splits.a
}

export const load: LayoutServerLoad = async ({ cookies, url }) => {
  const pathname = url.pathname.replace(/\/+$/, '')

  let splitTest = entryPoints[pathname]
  if (!splitTest) return

  try {
    const funnel = getVarient(splitTest.varients)
    return {
      funnel
    }
  } catch (e) {
    //handle the error the way you like or you can check the probabilities at compile time
  }
}

Finally, we want to make sure we cache the variant so that upon subsequent requests, we maintain the same variant on the client. We can just use a session cookie for this:

src/routes/(marketing)/+layout.server.ts

import type { LayoutServerLoad } from './$types';
import { splits, getVarient } from '$lib/splits';
import { funnels } from '$lib/funnel';
import type { PathName } from '$lib/funnel';
import type { SplitTest } from '$lib/splits';

type EntryPoints = Record<PathName, SplitTest>
const entryPoints: EntryPoints = {
  '/salespage': splits.a,
  '/checkout': splits.a
}

export const load: LayoutServerLoad = async ({ cookies, url }) => {
  let funnelId = cookies.get('funnelId')
  if (funnelId) return {
    funnel: funnels[funnelId]
  }

  const pathname = url.pathname.replace(/\/+$/, '')

  let splitTest = entryPoints[pathname]
  if (!splitTest) return

  try {
    const funnel = getVarient(splitTest.varients)
    cookies.set('funnelId', funnel.id)
    return {
      funnel
    }
  } catch (e) {
    //handle the error the way you like
  }
}

On the client, we need to create a pattern that will allow us to consume the split test data and render the pages accordingly. To start, we create the pages referenced in our funnel data ‘salespage/+page.svelte', ‘upsell/+page.svelte’, and checkout/+page.svelte' in the ‘(marketing)’ directory. For each of these pages, we can grab the FunnelElement data for the current page/variant, and if it does not exist, pass a default to our template:

+page.svelte

<script lang="ts">
  import { page } from "$app/stores";
  import type { FunnelElement } from "$lib/funnel";
  import type { PageData } from "./$types";

  export let data: PageData;

  const fallback: FunnelElement = {
    varient: 0,
    cta: {
      innerText: "Default Cta",
      href: "/checkout",
    },
  };

  const funnelData = data?.funnel?.data[$page.url.pathname] || fallback;
</script>

<h1>{$page.url.pathname}</h1>
<h2>Funnel Id: {data?.funnel?.id}</h2>

{#if funnelData.varient === 0}
  <h2>Page varient 0</h2>
  <p>call to action:</p>
  <a href={funnelData.cta.href}>{funnelData.cta.innerText}</a>
{:else if funnelData.varient === 1}
  <h2>Page varient 1</h2>
  <p>call to action:</p>
  <a href={funnelData.cta.href}>{funnelData.cta.innerText}</a>
{/if}

<p>{JSON.stringify(funnelData)}</p>

<p>funnel: {JSON.stringify(data?.funnel)}</p>

And that's it! We have a basic implementation that will allow us to run split tests on any combination of marketing pages and variants (repo here ). Of course, we would want to keep track of our data in a tool like Heap or GA , and route site crawlers to our default variants. Additionally, it would be best in most cases to keep all hard-coded data in this example in a Headless CMS or database of some kind, but I'll leave those ideas as an exercise for the reader.