Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(aci): schema validation for DataCondition.comparison #83457

Merged
merged 5 commits into from
Jan 15, 2025

Conversation

cathteng
Copy link
Member

Add schema validation for DataCondition's comparison JSONField. We need this to be absolutely sure the comparison field contains the correct information to be able to evaluate the condition in processors.

This PR adds the schema validation for the AgeComparison condition as an example.

@cathteng cathteng requested a review from saponifi3d January 14, 2025 22:43
@cathteng cathteng requested a review from a team as a code owner January 14, 2025 22:43
@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Jan 14, 2025
@@ -55,6 +55,11 @@ def bulk_get_query_object(data_sources) -> dict[int, T | None]:


class DataConditionHandler(Generic[T]):
@staticmethod
def comparison_json_schema() -> dict[str, Any]:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure how to have an abstract static property 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

@cathteng cathteng Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think i assumed we don't want to initialize the handler, but we can! added abstract property

Copy link

codecov bot commented Jan 14, 2025

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
23527 3 23524 271
View the top 2 failed tests by shortest run time
tests.symbolicator.test_unreal_full.SymbolicatorUnrealIntegrationTest::test_unreal_crash_with_attachments
Stack Traces | 0.076s run time
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "auth_user_username_key"#x1B[0m
#x1B[1m#x1B[31mE   DETAIL:  Key (username)=(admin@localhost) already exists.#x1B[0m

#x1B[33mDuring handling of the above exception, another exception occurred:#x1B[0m
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:77: in inner
    raise_the_exception(self.db, e)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:75: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:18: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:93: in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   psycopg2.errors.UniqueViolation: UniqueViolation('duplicate key value violates unique constraint "auth_user_username_key"\nDETAIL:  Key (username)=(admin@localhost) already exists.\n')#x1B[0m
#x1B[1m#x1B[31mE   SQL: INSERT INTO "auth_user" ("password", "last_login", "username", "first_name", "email", "is_staff", "is_active", "is_unclaimed", "is_superuser", "is_managed", "is_sentry_app", "is_password_expired", "last_password_change", "flags", "session_nonce", "date_joined", "last_active", "avatar_type", "avatar_url") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "auth_user"."id"#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[31mtests/symbolicator/test_unreal_full.py#x1B[0m:42: in initialize
    self.project.update_option("sentry:builtin_symbol_sources", [])
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:90: in project
    name="Bar", slug="bar", teams=[self.team], fire_project_created=True
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:80: in team
    team = self.create_team(organization=self.organization, name="foo", slug="foo")
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:75: in organization
    return self.create_organization(name="baz", slug="baz", owner=self.user)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:64: in user
    return self.create_user(
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:262: in create_user
    return Factories.create_user(*args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/factories.py#x1B[0m:903: in create_user
    user.save()
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:158: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../users/models/user.py#x1B[0m:225: in save
    result = super().save(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../contrib/auth/base_user.py#x1B[0m:62: in save
    super().save(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:892: in save
    self.save_base(
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:158: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:998: in save_base
    updated = self._save_table(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:1161: in _save_table
    results = self._do_insert(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:1202: in _do_insert
    return manager._insert(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/manager.py#x1B[0m:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/query.py#x1B[0m:1847: in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:1836: in execute_sql
    cursor.execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:122: in execute
    return super().execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../site-packages/sentry_sdk/utils.py#x1B[0m:1858: in runner
    return original_function(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:79: in execute
    return self._execute_with_wrappers(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
#x1B[1m#x1B[.../sentry/testutils/hybrid_cloud.py#x1B[0m:133: in __call__
    return execute(*params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:100: in _execute
    with self.db.wrap_database_errors:
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/db/utils.py#x1B[0m:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:77: in inner
    raise_the_exception(self.db, e)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:75: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:18: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:93: in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   django.db.utils.IntegrityError: UniqueViolation('duplicate key value violates unique constraint "auth_user_username_key"\nDETAIL:  Key (username)=(admin@localhost) already exists.\n')#x1B[0m
#x1B[1m#x1B[31mE   SQL: INSERT INTO "auth_user" ("password", "last_login", "username", "first_name", "email", "is_staff", "is_active", "is_unclaimed", "is_superuser", "is_managed", "is_sentry_app", "is_password_expired", "last_password_change", "flags", "session_nonce", "date_joined", "last_active", "avatar_type", "avatar_url") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "auth_user"."id"#x1B[0m
tests.symbolicator.test_unreal_full.SymbolicatorUnrealIntegrationTest::test_unreal_apple_crash_with_attachments
Stack Traces | 13.4s run time
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "auth_user_username_key"#x1B[0m
#x1B[1m#x1B[31mE   DETAIL:  Key (username)=(admin@localhost) already exists.#x1B[0m

#x1B[33mDuring handling of the above exception, another exception occurred:#x1B[0m
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:77: in inner
    raise_the_exception(self.db, e)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:75: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:18: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:93: in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   psycopg2.errors.UniqueViolation: UniqueViolation('duplicate key value violates unique constraint "auth_user_username_key"\nDETAIL:  Key (username)=(admin@localhost) already exists.\n')#x1B[0m
#x1B[1m#x1B[31mE   SQL: INSERT INTO "auth_user" ("password", "last_login", "username", "first_name", "email", "is_staff", "is_active", "is_unclaimed", "is_superuser", "is_managed", "is_sentry_app", "is_password_expired", "last_password_change", "flags", "session_nonce", "date_joined", "last_active", "avatar_type", "avatar_url") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "auth_user"."id"#x1B[0m

#x1B[33mThe above exception was the direct cause of the following exception:#x1B[0m
#x1B[1m#x1B[31mtests/symbolicator/test_unreal_full.py#x1B[0m:42: in initialize
    self.project.update_option("sentry:builtin_symbol_sources", [])
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:90: in project
    name="Bar", slug="bar", teams=[self.team], fire_project_created=True
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:80: in team
    team = self.create_team(organization=self.organization, name="foo", slug="foo")
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:75: in organization
    return self.create_organization(name="baz", slug="baz", owner=self.user)
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/utils/functional.py#x1B[0m:47: in __get__
    res = instance.__dict__[self.name] = self.func(instance)
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:64: in user
    return self.create_user(
#x1B[1m#x1B[.../sentry/testutils/fixtures.py#x1B[0m:262: in create_user
    return Factories.create_user(*args, **kwargs)
#x1B[1m#x1B[.../hostedtoolcache/Python/3.13.1....../x64/lib/python3.13/contextlib.py#x1B[0m:85: in inner
    return func(*args, **kwds)
#x1B[1m#x1B[.../sentry/testutils/factories.py#x1B[0m:903: in create_user
    user.save()
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:158: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[.../users/models/user.py#x1B[0m:225: in save
    result = super().save(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../contrib/auth/base_user.py#x1B[0m:62: in save
    super().save(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:892: in save
    self.save_base(
#x1B[1m#x1B[.../sentry/silo/base.py#x1B[0m:158: in override
    return original_method(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:998: in save_base
    updated = self._save_table(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:1161: in _save_table
    results = self._do_insert(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/base.py#x1B[0m:1202: in _do_insert
    return manager._insert(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/manager.py#x1B[0m:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/models/query.py#x1B[0m:1847: in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
#x1B[1m#x1B[31m.venv/lib/python3.13.../models/sql/compiler.py#x1B[0m:1836: in execute_sql
    cursor.execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:122: in execute
    return super().execute(sql, params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../site-packages/sentry_sdk/utils.py#x1B[0m:1858: in runner
    return original_function(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:79: in execute
    return self._execute_with_wrappers(
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:92: in _execute_with_wrappers
    return executor(sql, params, many, context)
#x1B[1m#x1B[.../sentry/testutils/hybrid_cloud.py#x1B[0m:133: in __call__
    return execute(*params)
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:100: in _execute
    with self.db.wrap_database_errors:
#x1B[1m#x1B[31m.venv/lib/python3.13.../django/db/utils.py#x1B[0m:91: in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
#x1B[1m#x1B[31m.venv/lib/python3.13.../db/backends/utils.py#x1B[0m:105: in _execute
    return self.cursor.execute(sql, params)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:77: in inner
    raise_the_exception(self.db, e)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:75: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:18: in inner
    return func(self, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:93: in inner
    raise type(e)(f"{e!r}\nSQL: {sql}").with_traceback(e.__traceback__)
#x1B[1m#x1B[.../db/postgres/decorators.py#x1B[0m:91: in inner
    return func(self, sql, *args, **kwargs)
#x1B[1m#x1B[.../db/postgres/base.py#x1B[0m:84: in execute
    return self.cursor.execute(sql, clean_bad_params(params))
#x1B[1m#x1B[31mE   django.db.utils.IntegrityError: UniqueViolation('duplicate key value violates unique constraint "auth_user_username_key"\nDETAIL:  Key (username)=(admin@localhost) already exists.\n')#x1B[0m
#x1B[1m#x1B[31mE   SQL: INSERT INTO "auth_user" ("password", "last_login", "username", "first_name", "email", "is_staff", "is_active", "is_unclaimed", "is_superuser", "is_managed", "is_sentry_app", "is_password_expired", "last_password_change", "flags", "session_nonce", "date_joined", "last_active", "avatar_type", "avatar_url") VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s) RETURNING "auth_user"."id"#x1B[0m
View the full list of 1 ❄️ flaky tests
tests.sentry.api.endpoints.test_organization_events_trends_v2.OrganizationEventsTrendsStatsV2EndpointTest::test_two_projects_same_transaction_split_queries

Flake rate in main: 20.00% (Passed 12 times, Failed 3 times)

Stack Traces | 6.26s run time
#x1B[1m#x1B[.../api/endpoints/test_organization_events_trends_v2.py#x1B[0m:414: in test_two_projects_same_transaction_split_queries
    assert len(trends_call_args_data_1[f"{project1.id},foo bar*"]) > 0
#x1B[1m#x1B[31mE   KeyError: '4555389477584900,foo bar*'#x1B[0m

To view more test analytics, go to the Test Analytics Dashboard
📢 Thoughts on this report? Let us know!

Comment on lines 128 to 151
@receiver(pre_save, sender=DataCondition)
def enforce_comparison_schema(sender, instance: DataCondition, **kwargs):
from jsonschema import ValidationError, validate

condition_type = Condition(instance.type)
if condition_type in condition_ops:
# don't enforce schema for default ops, this can be any type
return

try:
handler = condition_handler_registry.get(instance.type)
except NoRegistrationExistsError:
logger.exception(
"No registration exists for condition",
extra={"type": instance.type, "id": instance.id},
)
return None

schema = handler.comparison_json_schema

try:
validate(instance.comparison, schema)
except ValidationError as e:
raise ValidationError(f"Invalid config: {e.message}")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the important part

"time": "hour",
}
assert dc.condition_result is True
assert dc.condition_group == dcg

def test_json_schema(self):
with pytest.raises(ValidationError):
self.dc.comparison.update({"time": "asdf"})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the validation error is raised on save, right? maybe you can put the update outside of the pytest raise here to make that more clear

from jsonschema import ValidationError, validate

condition_type = Condition(instance.type)
if condition_type in condition_ops:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit but it would be nice for condition ops to be capitalized

Copy link
Contributor

@saponifi3d saponifi3d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - agree with colleen's nits :)

@cathteng cathteng enabled auto-merge (squash) January 15, 2025 19:01
@cathteng cathteng merged commit 47d75ae into master Jan 15, 2025
48 checks passed
@cathteng cathteng deleted the cathy/aci/dc-comparison-schema-validation branch January 15, 2025 19:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Scope: Backend Automatically applied to PRs that change backend components
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants