[WIP] - AB Experiments (Balancy)

Introduction

• AB Experiments are set up on the Balancy Dashboard. Ensure your Balancy instance is set up and you can access your Balancy project.
• LionSDK will automatically fire theab_cohort event to LionAnalytics, allowing us to analyze the experiment results
• This document explains the requirements using an example use-case.

Pre-requisite

• Install and setup Balancy in Unity project as explained here
• Install LiveOps plugin in Balancy game dashboard.

At this point, you will be able to fetch remote configs from Balancy via LionSDK (Remote Config) package.

Requirements

• Create an experiment with an experiment name by going to your Balancy game dashboard and follow the experiment naming convention
  • Naming Convention → {game_code}_{platform}_{testName}
    • game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode.
    • platform:
      • ios for iOS
      • and for Android
      • all for tests run on both platforms simultaneously (start and end dates should be equal).
    • testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.

Example Use-Case

• Let’s suppose there is a game with game code csc runs an experiment to evaluate the impact of different interstitial ad timings. The experiment is named csc_and_newInterTimer, with the following variants and configurations:

  Variants	inter\_between\_time	inter\_start\_time
  baseline	90	90
  aggressive	60	60
  passive	120	120

• Each variant receives different values for the experiment’s inter_between_time and inter_start_time remote configs.

Requirements

• Create an experiment with an experiment name following the experiment naming convention

  • Naming Convention → {game_code}_{platform}_{testName}
    • game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode.
    • platform:
      • ios for iOS
      • and for Android
      • all for tests run on both platforms simultaneously (start and end dates should be equal).
    • testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.
  • Example → csc_and_newInterTimer

  

• Create templates/remote configs that the experiment will override for each experiment cohort

  

• Setup the override values for each experiment cohort