-
Notifications
You must be signed in to change notification settings - Fork 301
Stable Config file: target system properties in process_arguments and support template variables in YamlParser #8690
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
BenchmarksStartupLoadDacapo |
dd-java-agent/src/main/java/datadog/trace/bootstrap/AgentBootstrap.java
Outdated
Show resolved
Hide resolved
components/yaml/src/test/groovy/datadog/yaml/YamlParserTest.groovy
Outdated
Show resolved
Hide resolved
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hey @mtoffl01
I feel there is a mix of concerns about how the CLIHelper
changed.
As it’s a common composent, we would expect him to keep providing the JVM arguments. (-Dkey=value
being a process argument in itself after all)
While I think that parsing JVM properties is interesting (and needed), it should not replace the process arguments API.
So what about having:
- one way to retrieve arguments as a list of string (as before)
- one way of querying JVM properties (get the key value if any)
- but still do the parsing in one pass
Overall, I also think the common component needs:
- A complete Javadoc to define its expected capability / behavior,
- Some tests in order to check edge case behavior (like property redefinition, empty property value, etc...).
What do you thing about it?
And my mid-term goal is to make this component evolves to a system
/ env
component that will be provide any kind of information about the execution context (JDK version, vendor, native-image, etc...).
throw new IOException("Empty environment variable name in template"); | ||
} | ||
String value = System.getenv(envVar.toUpperCase()); | ||
if (value == null || value.isEmpty()) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One question - will users ever want to substitute a variable with an empty string?
Currently we don't support this, because the variable will be replaced with UNDEFINED
- but a user might want to have some kind of optional variable that may have a value or optionally be the empty string...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressing in a separate PR: #8759
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a question about whether we should allow users to set variables to the empty string or not
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would avoid creating one-liner helpers in internal-api
.
They can be replaced by their equivalent static method calls from JDK.
static writeFileRaw(Path filePath, String data) { | ||
StandardOpenOption[] openOpts = [StandardOpenOption.WRITE] as StandardOpenOption[] | ||
Files.write(filePath, data.getBytes(), openOpts) | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is that useful to create a dedicated method for a single call which can already be written: Files.write(path, data.getBytes())
?
Default open options looks good enough for the testing, aren't they?
And I would not recommend re-using this method as it hide the encoding / charset to used (which is usually not recommended).
try { | ||
return Files.createTempFile("testFile_", ".yaml") | ||
} catch (IOException e) { | ||
println "Error creating file: ${e.message}" | ||
e.printStackTrace() | ||
return null | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Similarly, this is one-liner helper.
Checking the exception is not required is groovy so you can directly write Files.createTempFile()
in your test, and the exception handling won't be useful as you will certainly end up with another exception (NullPoienterException) if something fails.
Additionally, Spock and test framework usually provide tools for temporary files (for example @TempDir
for Spock: https://spockframework.org/spock/docs/2.3/all_in_one.html#_temp_dir)
| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.google.cloud:google-cloud-logging](https://github.com/googleapis/java-logging) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `3.22.2` -> `3.22.3` | | [com.google.cloud:google-cloud-datastore](https://github.com/googleapis/java-datastore) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.28.0` -> `2.28.1` | | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.48.2` -> `1.49.0` | | [software.amazon.awssdk:sdk-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:sqs](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:dynamodb-enhanced](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:dynamodb](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:aws-core](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:bom](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | | [software.amazon.awssdk:auth](https://aws.amazon.com/sdkforjava) | dependencies | misk/gradle/libs.versions.toml | gradle | patch | `2.31.36` -> `2.31.37` | --- ### Release Notes <details> <summary>googleapis/java-logging (com.google.cloud:google-cloud-logging)</summary> ### [`v3.22.3`](https://github.com/googleapis/java-logging/blob/HEAD/CHANGELOG.md#3223-2025-05-06) ##### Bug Fixes - **deps:** Update the Java code generator (gapic-generator-java) to 2.56.3 ([844f4fa](googleapis/java-logging@844f4fa)) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.46.3 ([#​1801](googleapis/java-logging#1801)) ([d7aa7bc](googleapis/java-logging@d7aa7bc)) - Update dependency com.google.cloud:sdk-platform-java-config to v3.47.0 ([#​1803](googleapis/java-logging#1803)) ([5967ffe](googleapis/java-logging@5967ffe)) - Update googleapis/sdk-platform-java action to v2.57.0 ([#​1804](googleapis/java-logging#1804)) ([e9a27ec](googleapis/java-logging@e9a27ec)) </details> <details> <summary>googleapis/java-datastore (com.google.cloud:google-cloud-datastore)</summary> ### [`v2.28.1`](https://github.com/googleapis/java-datastore/blob/HEAD/CHANGELOG.md#2281-2025-05-06) ##### Dependencies - Update dependency com.google.cloud:sdk-platform-java-config to v3.47.0 ([#​1841](googleapis/java-datastore#1841)) ([ac393e6](googleapis/java-datastore@ac393e6)) - Update googleapis/sdk-platform-java action to v2.57.0 ([#​1842](googleapis/java-datastore#1842)) ([0745906](googleapis/java-datastore@0745906)) </details> <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.49.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.49.0): 1.49.0 ### Components #### Configuration at Runtime - ✨ Add process tags as list to remote config payload ([#​8705](DataDog/dd-trace-java#8705) - [@​amarziali](https://github.com/amarziali)) #### Continuous Integration Visibility - 🐛 Add span propagation for Pekko scheduled tasks ([#​8765](DataDog/dd-trace-java#8765) - [@​nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Update test.retry_reason to use full name of the feature ([#​8689](DataDog/dd-trace-java#8689) - [@​daniel-mohedano](https://github.com/daniel-mohedano)) - 🧹 Remove unused TestEventsHandler methods ([#​8674](DataDog/dd-trace-java#8674) - [@​nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) #### Dynamic Instrumentation - 🐛 Fix exclude identifiers normalization ([#​8742](DataDog/dd-trace-java#8742) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Make source file tracking asynchronous ([#​8684](DataDog/dd-trace-java#8684) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Add scope filtering for symbol extraction ([#​8676](DataDog/dd-trace-java#8676) - [@​jpbempel](https://github.com/jpbempel)) - ✨ Add support for [@​key](https://github.com/key) and [@​value](https://github.com/value) for Map filtering ([#​8669](DataDog/dd-trace-java#8669) - [@​jpbempel](https://github.com/jpbempel)) #### Library Injection - ✨ Add system property to force injection of the tracing library even though multiple javaagents have been detected ([#​8697](DataDog/dd-trace-java#8697) - [@​cecile75](https://github.com/cecile75)) #### Metrics - ✨ Allow dogstatsd port to be configurable with DD_DOGSTATSD_PORT ([#​8693](DataDog/dd-trace-java#8693) - [@​randomanderson](https://github.com/randomanderson)) #### Profiling - ✨ Bump ddprof-java to 1.25.1 ([#​8750](DataDog/dd-trace-java#8750) - [@​jbachorik](https://github.com/jbachorik)) - 🐛 Remove cleanup-on-shutdown for temporary files ([#​8746](DataDog/dd-trace-java#8746) - [@​jbachorik](https://github.com/jbachorik)) - ✨⚡ Replace a regex-based SMAP parser with a hand-crafted one ([#​8730](DataDog/dd-trace-java#8730) - [@​jbachorik](https://github.com/jbachorik)) - ✨ Improve error reporting on profiler startup ([#​8714](DataDog/dd-trace-java#8714) - [@​jbachorik](https://github.com/jbachorik)) - ✨ Exclude ProxyLeakTask exception from exception profiling ([#​8666](DataDog/dd-trace-java#8666) - [@​jbachorik](https://github.com/jbachorik)) - ✨ Use jvmstat for JDKs 9+ programmatically ([#​8641](DataDog/dd-trace-java#8641) - [@​MattAlp](https://github.com/MattAlp)) #### Telemetry - ✨ Allow dogstatsd port to be configurable with DD_DOGSTATSD_PORT ([#​8693](DataDog/dd-trace-java#8693) - [@​randomanderson](https://github.com/randomanderson)) - 🐛 Fix appsec.waf.requests telemetry metric ([#​8644](DataDog/dd-trace-java#8644) - [@​jandro996](https://github.com/jandro996)) #### Tracer core - ✨ Exclude jackson afterburner dynamic classes from instrumentation ([#​8747](DataDog/dd-trace-java#8747) - [@​amarziali](https://github.com/amarziali)) - ✨ Introduce Java 8 bytecode bridge for instrumentation API ([#​8736](DataDog/dd-trace-java#8736) - [@​PerfectSlayer](https://github.com/PerfectSlayer)) - ⚡🧹 Use byte-buddy classes optimized for Java8+ ([#​8735](DataDog/dd-trace-java#8735) - [@​mcculls](https://github.com/mcculls)) - 🐛 Do not set the hibernate or datanucleus span service name when disabled ([#​8727](DataDog/dd-trace-java#8727) - [@​ygree](https://github.com/ygree)) - ✨ Update bytebuddy and ASM to support JDK 24 ([#​8720](DataDog/dd-trace-java#8720) - [@​sarahchen6](https://github.com/sarahchen6)) - 🐛 Turn off JDK socket support by default ([#​8715](DataDog/dd-trace-java#8715) - [@​mcculls](https://github.com/mcculls)) - 🐛 Log warning when trace buffer overflow occurs ([#​8712](DataDog/dd-trace-java#8712) - [@​ygree](https://github.com/ygree)) - ✨🧪 Introducing an internal integration name ([#​8708](DataDog/dd-trace-java#8708) - [@​amarziali](https://github.com/amarziali)) - ✨ Add process tags to client stats payload ([#​8704](DataDog/dd-trace-java#8704) - [@​amarziali](https://github.com/amarziali)) - ✨ Collect process tags for tracing ([#​8698](DataDog/dd-trace-java#8698) - [@​amarziali](https://github.com/amarziali)) - ✨ Stable Config file: target system properties in process_arguments and support template variables in YamlParser ([#​8690](DataDog/dd-trace-java#8690) - [@​mtoffl01](https://github.com/mtoffl01)) - ✨⚡ Use prefix trie for proxy ignores ([#​8678](DataDog/dd-trace-java#8678) - [@​amarziali](https://github.com/amarziali)) - ✨ Allow agent to be automatically injected when running aside Log4J patch agent ([#​8648](DataDog/dd-trace-java#8648) - [@​paullegranddc](https://github.com/paullegranddc)) - ✨ Use jvmstat for JDKs 9+ programmatically ([#​8641](DataDog/dd-trace-java#8641) - [@​MattAlp](https://github.com/MattAlp)) #### Tracer internal logging - 🐛 Delete print line ([#​8686](DataDog/dd-trace-java#8686) - [@​sarahchen6](https://github.com/sarahchen6)) ### Instrumentations #### Akka instrumentation - 🐛 Handle reentrant scope cleanup in Akka/Pekko actor instrumentations ([#​8722](DataDog/dd-trace-java#8722) - [@​mcculls](https://github.com/mcculls)) #### Apache Spark instrumentation - ✨ Use OpenLineage root parent information to generate trace id ([#​8726](DataDog/dd-trace-java#8726) - [@​mobuchowski](https://github.com/mobuchowski)) - ✨ Spark job cancellation no longer marks application as failed ([#​8701](DataDog/dd-trace-java#8701) - [@​paul-laffon-dd](https://github.com/paul-laffon-dd)) #### JDBC instrumentation - 💡 Add support for sybase tds jdbc driver ([#​8764](DataDog/dd-trace-java#8764) - [@​amarziali](https://github.com/amarziali)) #### Kotlin instrumentation - 🐛 Take defensive copy of parent scope stack when closing nested coroutines ([#​8749](DataDog/dd-trace-java#8749) - [@​mcculls](https://github.com/mcculls)) #### Reactor instrumentation - ✨⚡ Do not inspect reactor context when not needed ([#​8745](DataDog/dd-trace-java#8745) - [@​amarziali](https://github.com/amarziali)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://github.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: 795f347ae34d056efc1194c2f606cee7bca1beea
What Does This Do
This PR adds support for matching system property keys in the Stable Configuration selector, even when those properties include values. For example, a selector with
-Darg1
will now correctly match a Java process started with-Darg1=value
.It also introduces support for template variables in the Stable Configuration file schema. These variables can resolve values from process arguments and environment variables. (See reference for more details.)
Notes & Limitations:
Scope Limitation: This change restricts
process_arguments
selectors and template variables to System Properties only.Required Format: System Properties must be referenced using their full -D prefix, without specifying a value (e.g., use
-Dkey
, notkey
nor-Dkey=value
).Motivation
Stable Config users must be able to target services based on the presence of a specific System Property, regardless of its value. For instance,
-Darg1
should match any process started with-Darg1=abcd
,-Darg1=xyz
, etc.Currently, our only known use case for
process_arguments
selectors involves System Properties. If a need arises to support additional types of JVM arguments in the future, we can revisit and expand this functionality.Additional Notes
Contributor Checklist
type:
and (comp:
orinst:
) labels in addition to any usefull labelsclose
,fix
or any linking keywords when referencing an issue.Use
solves
instead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]