Rails 的 A/B 测试
:maple_leaf: A/B testing for Rails
Uses Bayesian statistics to evaluate results so you don’t need to choose a sample size ahead of time.
Add this line to your application’s Gemfile:
gem "field_test"Run:
rails generate field_test:install
rails db:migrateAnd mount the dashboard in your config/routes.rb:
mount FieldTest::Engine, at: "field_test"Be sure to secure the dashboard in production.
Add an experiment to config/field_test.yml.
experiments:
button_color:
variants:
- red
- green
- blueRefer to it in controllers, views, and mailers.
button_color = field_test(:button_color)To make testing easier, you can specify a variant with query parameters
http://localhost:3000/?field_test[button_color]=greenWhen someone converts, record it with:
field_test_converted(:button_color)When an experiment is over, specify a winner:
experiments:
button_color:
winner: greenAll calls to field_test will now return the winner, and metrics will stop being recorded.
You can keep returning the variant for existing participants after a winner is declared:
experiments:
button_color:
winner: green
keep_variant: trueYou can also close an experiment to new participants without declaring a winner while still recording metrics for existing participants:
experiments:
button_color:
closed: trueCalls to field_test for new participants will return the control, and they won’t be added to the experiment.
You can get the list of experiments and variants for a user with:
field_test_experimentsFor JavaScript and native apps, add calls to your normal endpoints.
class CheckoutController
Different button colors
for the landing page.By default, variants are given the same probability of being selected. Change this with:
experiments:
button_color:
variants:
- red
- blue
weights:
- 85
- 15To help with GDPR compliance, you can switch from cookies to anonymity sets for anonymous visitors. Visitors with the same IP mask and user agent are grouped together.
cookies: falseIf the dashboard gets slow, you can make it faster with:
cache: trueThis will use the Rails cache to speed up winning probability calculations.
If you need more precision, set:
precision: 1You can set multiple goals for an experiment to track conversions at different parts of the funnel. First, run:
rails generate field_test:events
rails db:migrateAnd add to your config:
experiments:
button_color:
goals:
- signed_up
- orderedSpecify a goal during conversion with:
field_test_converted(:button_color, goal: "ordered")The results for all goals will appear on the dashboard.
You may also want to send experiment data as properties to other analytics platforms like Segment, Amplitude, and Ahoy. Get the list of experiments and variants with:
field_test_experimentsYou can configure Field Test to use Ahoy’s visitor token instead of creating its own:
class ApplicationController (user) { user.admin? } do
mount FieldTest::Engine, at: "field_test"
endSet the following variables in your environment or an initializer.
ENV["FIELD_TEST_USERNAME"] = "moonrise"
ENV["FIELD_TEST_PASSWORD"] = "kingdom"Assign a specific variant to a user with:
experiment = FieldTest::Experiment.find(:button_color)
experiment.variant(participant, variant: "green")You can also change a user’s variant from the dashboard.
To associate models with field test memberships, use:
class User < ApplicationRecord
has_many :field_test_memberships, class_name: "FieldTest::Membership", as: :participant
endNow you can do:
user.field_test_membershipsA huge thanks to Evan Miller for deriving the Bayesian formulas.
View the changelog
Everyone is encouraged to help improve this project. Here are a few ways you can help:
To get started with development:
git clone https://github.com/ankane/field_test.git
cd field_test
bundle install
bundle exec rake compile
bundle exec rake test暂无开放 Issues,或尚未同步最近议题。