Serialize span events via a dedicated field

[marcotc]

What does this PR do?
Before this PR, tracing span events were serialized as a JSON string in the span tag events. This causes issues with size limit, as tags can only be 5000 characters. This limit is problematic when reporting multiple span events, or when reporting large data in a span events, like a stack trace.

This PR uses a new top-level span field, span_events, which supports the full span event representation, without sizing limitation or artificial stringification that was required in the JSON serialization from before.

Because users can already use span events through the events tag, and agent support for the top-level field is scheduled for a future agent version, this change check with the agent before sending span events it in the new format.

Change log entry
Remove tracing Span Events serialization limitations.

How to test the change?
There are system-tests, as well as unit tests for this change.

[github-actions]

👋 Hey @marcotc, please fill "Change log entry" section in the pull request description.

If changes need to be present in CHANGELOG.md you can state it this way

**Change log entry**

Yes. A brief summary to be placed into the CHANGELOG.md

(possible answers Yes/Yep/Yeah)

Or you can opt out like that

**Change log entry**

None.

(possible answers No/Nope/None)

^{Visited at: 2025-01-15 16:58:45 UTC}

[datadog-datadog-prod-us1]

⚪ Code Quality Violation

Suggested change

	allow(trace_encoder).to receive(:encode_trace).with(encoder, traces[0], native_events_supported).and_return('1')
	allow(trace_encoder).to receive(:encode_trace).with(encoder, traces.first, native_events_supported).and_return('1')

Improve readability with first (...read more)

This rule encourages the use of first and last methods over array indexing to access the first and last elements of an array, respectively. The primary reason behind this rule is to improve code readability. Using first and last makes it immediately clear that you are accessing the first or last element of the array, which might not be immediately obvious with array indexing, especially for developers who are new to Ruby.

The use of these methods also helps to make your code more idiomatic, which is a crucial aspect of writing effective Ruby code. Idiomatic code is easier to read, understand, and maintain. It also tends to be more efficient, as idioms often reflect patterns that are optimized for the language.

To adhere to this rule, replace the use of array indexing with first or last methods when you want to access the first and last elements of an array. For instance, instead of arr[0] use arr.first and instead of arr[-1] use arr.last. However, note that this rule should be applied only when reading values. When modifying the first or last elements, array indexing should still be used. For example, arr[0] = 'new_value' and arr[-1] = 'new_value'.

    

[datadog-datadog-prod-us1]

Datadog Report

Branch report: span-events-writer
Commit report: 148f1b0
Test service: dd-trace-rb

✅ 0 Failed, 22228 Passed, 1477 Skipped, 7m 5.72s Total Time

[pr-commenter]

Benchmarks

Benchmark execution time: 2025-01-15 00:15:17

Comparing candidate commit 148f1b0 in PR branch span-events-writer with baseline commit 3dddec7 in branch steep-2.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 31 metrics, 2 unstable metrics.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.74%. Comparing base (3dddec7) to head (148f1b0).

Additional details and impacted files

@@             Coverage Diff             @@
##           steep-2    #4279      +/-   ##
===========================================
+ Coverage    97.71%   97.74%   +0.02%     
===========================================
  Files         1356     1357       +1     
  Lines        82494    82590      +96     
  Branches      4221     4227       +6     
===========================================
+ Hits         80611    80724     +113     
+ Misses        1883     1866      -17     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.