Skip to content

Conversation

@kappa90
Copy link
Contributor

@kappa90 kappa90 commented Oct 31, 2025

Problem

This PR introduces a new MaxTool to create experiments. It builds on top of the create_feature_flag MaxTool with multivariate flag support.

Changes

  • Added a new CreateExperimentTool that allows creating experiments via PostHog AI

How did you test this code?

New tests + evals

Changelog: (features only) Is this feature complete?

Yes, this feature is complete and ready to be included in the changelog.

Copy link
Contributor Author

kappa90 commented Oct 31, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@kappa90 kappa90 requested review from a team October 31, 2025 16:58
@kappa90 kappa90 marked this pull request as ready for review October 31, 2025 16:58
@github-actions
Copy link
Contributor

github-actions bot commented Oct 31, 2025

Size Change: 0 B

Total Size: 3.34 MB

ℹ️ View Unchanged
Filename Size
frontend/dist/toolbar.js 3.34 MB

compressed-size-action

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

10 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Examples:
- "Create an experiment to test the new checkout flow"
- "Set up an A/B test for our pricing page redesign"
- "Create an experiment called 'homepage-cta-test' to test different call-to-action buttons
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

syntax: missing closing quote - will cause syntax error

Suggested change
- "Create an experiment called 'homepage-cta-test' to test different call-to-action buttons
- "Create an experiment called 'homepage-cta-test' to test different call-to-action buttons"
Prompt To Fix With AI
This is a comment left during a code review.
Path: products/experiments/backend/max_tools.py
Line: 41:41

Comment:
**syntax:** missing closing quote - will cause syntax error

```suggestion
- "Create an experiment called 'homepage-cta-test' to test different call-to-action buttons"
```

How can I resolve this? If you propose a fix, please make it concise.
TODO_WRITE = "todo_write"
FILTER_REVENUE_ANALYTICS = "filter_revenue_analytics"
CREATE_FEATURE_FLAG = "create_feature_flag"
CREATE_EXPERIMENT = "create_experiment"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: posthog/schema.py is autogenerated by pnpm run schema:build and should not be manually modified. regenerate this file using the command instead of manual edits

Context Used: Rule from dashboard - Do not manually modify the posthog/schema.py file as it is autogenerated by the `pnpm run schema:b... (source)

Prompt To Fix With AI
This is a comment left during a code review.
Path: posthog/schema.py
Line: 285:285

Comment:
**style:** `posthog/schema.py` is autogenerated by `pnpm run schema:build` and should not be manually modified. regenerate this file using the command instead of manual edits

**Context Used:** Rule from `dashboard` - Do not manually modify the `posthog/schema.py` file as it is autogenerated by the `pnpm run schema:b... ([source](https://app.greptile.com/review/custom-context?memory=192de143-59d7-412f-bd71-fe5cb8f161dd))

How can I resolve this? If you propose a fix, please make it concise.
@kappa90 kappa90 force-pushed the 10-31-feat_ph-ai_create_experiments_max_tool branch from 16e71d9 to e85e00d Compare October 31, 2025 17:05
@kappa90 kappa90 force-pushed the 10-31-feat_ph-ai_create_multivariate_flags_max_tool branch from 4939dea to 3d8cb87 Compare October 31, 2025 17:05
@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

2 snapshot changes in total. 0 added, 2 modified, 0 deleted:

  • chromium: 0 added, 2 modified, 0 deleted (diff for shard 5)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

3 snapshot changes in total. 0 added, 3 modified, 0 deleted:

  • chromium: 0 added, 3 modified, 0 deleted (diff for shard 5)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

@posthog-bot
Copy link
Contributor

📸 UI snapshots have been updated

1 snapshot changes in total. 0 added, 1 modified, 0 deleted:

  • chromium: 0 added, 1 modified, 0 deleted (diff for shard 11)
  • webkit: 0 added, 0 modified, 0 deleted

Triggered by this commit.

👉 Review this PR's diff of snapshots.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants