Skip to content

Conversation

@KillerX
Copy link
Member

@KillerX KillerX commented Apr 22, 2024

This allows us to pass in the needed episode_ids and cut the query time by 2 orders of magnitude.

There are a few consequences:

  • SQLc config needed to be upgraded to v2
  • In order for SQLc to properly understand the function it needs a active SQL connection
  • The function returns are always "nullable" so we need to account for that in the Go code
  • Since the returns for the list variant is no longer identical we need a separate mapping

The logic is tested but I have not tested it yet with an running API.

This allows us to pass in the needed episode_ids and cut the query time by 2 orders of magnitude.
There are a few consequences:
* SQLc config needed to be upgraded to v2
* In order for SQLc to properly understand the function it needs a active SQL connection
* The function returns are always "nullable" so we need to account for that in the Go code
* Since the returns for the `list` variant is no longer identical we need a separate mapping
@KillerX KillerX requested a review from andreasgangso April 22, 2024 14:03
@KillerX
Copy link
Member Author

KillerX commented Apr 22, 2024

Original

image

New

image

json_tags_case_style: camel
overrides:
# For reference, when we need string-arrays (e.g. "usergroups" for a row on the materialized view)
# I think I had to do that manually for every column like this:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lol i think this is a remnant from my POC. can be removed

Copy link
Contributor

@andreasgangso andreasgangso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adding the db connection is fine with me although it's ofc nice to avoid it if we can.
the nullability is less comfortable but since it's only that function its not horrible.

  • maybe run once locally first.
  • try the out param thing, maybe it works without a connection

but i approve, its gefn if you want to merge this


-- +goose StatementBegin
CREATE OR REPLACE FUNCTION mediaitems_by_episodes(episodeids int[])
RETURNS TABLE
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe try OUT params, since this PR is mentioning that: sqlc-dev/sqlc#3175

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately you can't combine them: [42804] ERROR: RETURN cannot have a parameter in function with OUT parameters

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The other alternative would be to create a materialized view for the innermost SELECTS, then we could probably get away with a "normal" view.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now using "setof" option for returning from a function
This takes care both of needing a real DB connection, as well as nullable fields.

This takes care both of needing a real DB connection, as well as nullable fields.
@KillerX KillerX requested a review from andreasgangso April 23, 2024 11:32
@KillerX KillerX merged commit ec89e7a into master Apr 23, 2024
@KillerX KillerX deleted the fix/optimize-view branch April 23, 2024 11:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants