title | description | author | ms.author | ms.topic | ms.date |
---|---|---|---|---|---|
Migrate Splunk detection rules to Microsoft Sentinel | Microsoft Docs |
Learn how to identify, compare, and migrate your Splunk detection rules to Microsoft Sentinel built-in rules. |
limwainstein |
lwainstein |
how-to |
05/03/2022 |
This article describes how to identify, compare, and migrate your Splunk detection rules to Microsoft Sentinel built-in rules.
Microsoft Sentinel uses machine learning analytics to create high-fidelity and actionable incidents, and some of your existing detections may be redundant in Microsoft Sentinel. Therefore, don't migrate all of your detection and analytics rules blindly. Review these considerations as you identify your existing detection rules.
- Make sure to select use cases that justify rule migration, considering business priority and efficiency.
- Check that you understand Microsoft Sentinel rule types.
- Check that you understand the rule terminology.
- Review any rules that haven't triggered any alerts in the past 6-12 months, and determine whether they're still relevant.
- Eliminate low-level threats or alerts that you routinely ignore.
- Use existing functionality, and check whether Microsoft Sentinel’s built-in analytics rules might address your current use cases. Because Microsoft Sentinel uses machine learning analytics to produce high-fidelity and actionable incidents, it’s likely that some of your existing detections won’t be required anymore.
- Confirm connected data sources and review your data connection methods. Revisit data collection conversations to ensure data depth and breadth across the use cases you plan to detect.
- Explore community resources such as the SOC Prime Threat Detection Marketplace to check whether your rules are available.
- Consider whether an online query converter such as Uncoder.io might work for your rules.
- If rules aren’t available or can’t be converted, they need to be created manually, using a KQL query. Review the rules mapping to create new queries.
Learn more about best practices for migrating detection rules.
To migrate your analytics rules to Microsoft Sentinel:
-
Verify that you have a testing system in place for each rule you want to migrate.
-
Prepare a validation process for your migrated rules, including full test scenarios and scripts.
-
Ensure that your team has useful resources to test your migrated rules.
-
Confirm that you have any required data sources connected, and review your data connection methods.
-
-
Verify whether your detections are available as built-in templates in Microsoft Sentinel:
-
If the built-in rules are sufficient, use built-in rule templates to create rules for your own workspace.
In Microsoft Sentinel, go to the Configuration > Analytics > Rule templates tab, and create and update each relevant analytics rule.
For more information, see Detect threats out-of-the-box.
-
If you have detections that aren't covered by Microsoft Sentinel's built-in rules, try an online query converter, such as Uncoder.io to convert your queries to KQL.
Identify the trigger condition and rule action, and then construct and review your KQL query.
-
If neither the built-in rules nor an online rule converter is sufficient, you'll need to create the rule manually. In such cases, use the following steps to start creating your rule:
-
Identify the data sources you want to use in your rule. You'll want to create a mapping table between data sources and data tables in Microsoft Sentinel to identify the tables you want to query.
-
Identify any attributes, fields, or entities in your data that you want to use in your rules.
-
Identify your rule criteria and logic. At this stage, you may want to use rule templates as samples for how to construct your KQL queries.
Consider filters, correlation rules, active lists, reference sets, watchlists, detection anomalies, aggregations, and so on. You might use references provided by your legacy SIEM to understand how to best map your query syntax.
-
Identify the trigger condition and rule action, and then construct and review your KQL query. When reviewing your query, consider KQL optimization guidance resources.
-
-
-
Test the rule with each of your relevant use cases. If it doesn't provide expected results, you may want to review the KQL and test it again.
-
When you're satisfied, you can consider the rule migrated. Create a playbook for your rule action as needed. For more information, see Automate threat response with playbooks in Microsoft Sentinel.
Learn more about analytics rules:
- Create custom analytics rules to detect threats. Use alert grouping to reduce alert fatigue by grouping alerts that occur within a given timeframe.
- Map data fields to entities in Microsoft Sentinel to enable SOC engineers to define entities as part of the evidence to track during an investigation. Entity mapping also makes it possible for SOC analysts to take advantage of an intuitive [investigation graph (investigate-cases.md#use-the-investigation-graph-to-deep-dive) that can help reduce time and effort.
- Investigate incidents with UEBA data, as an example of how to use evidence to surface events, alerts, and any bookmarks associated with a particular incident in the incident preview pane.
- Kusto Query Language (KQL), which you can use to send read-only requests to your Log Analytics database to process data and return results. KQL is also used across other Microsoft services, such as Microsoft Defender for Endpoint and Application Insights.
This table helps you to clarify the concept of a rule in Microsoft Sentinel compared to Splunk.
Splunk | Microsoft Sentinel | |
---|---|---|
Rule type | • Scheduled • Real-time |
• Scheduled query • Fusion • Microsoft Security • Machine Learning (ML) Behavior Analytics |
Criteria | Define in SPL | Define in KQL |
Trigger condition | • Number of results • Number of hosts • Number of sources • Custom |
Threshold: Number of query results |
Action | • Add to triggered alerts • Log Event • Output results to lookup • And more |
• Create alert or incident • Integrates with Logic Apps |
Use these samples to compare and map rules from Splunk to Microsoft Sentinel in various scenarios.
SPL command | Description | KQL operator | KQL example |
---|---|---|---|
chart/ timechart |
Returns results in a tabular output for time-series charting. | render operator | `… |
dedup |
Removes subsequent results that match a specified criterion. | • distinct • summarize |
`… |
eval |
Calculates an expression. Learn about common eval commands. | extend | `T |
fields |
Removes fields from search results. | • project • project-away |
`T |
head/tail |
Returns the first or last N results. | top | `T |
lookup |
Adds field values from an external source. | • externaldata • lookup |
KQL example |
rename |
Renames a field. Use wildcards to specify multiple fields. | project-rename | `T |
rex |
Specifies group names using regular expressions to extract fields. | matches regex | `… |
search |
Filters results to results that match the search expression. | search | search "X" |
sort |
Sorts the search results by the specified fields. | sort | `T |
stats |
Provides statistics, optionally grouped by fields. Learn more about common stats commands. | summarize | KQL example |
mstats |
Similar to stats, used on metrics instead of events. | summarize | KQL example |
table |
Specifies which fields to keep in the result set, and retains data in tabular format. | project | `T |
top/rare |
Displays the most or least common values of a field. | top | `T |
transaction |
Groups search results into transactions. SPL example |
Example: row_window_session | KQL example |
eventstats |
Generates summary statistics from fields in your events and saves those statistics in a new field. SPL example |
Examples: • join • make_list • mv-expand |
KQL example |
streamstats |
Find the cumulative sum of a field. SPL example: `... |
streamstats sum(bytes) as bytes _ total | timechart` | row_cumsum |
anomalydetection |
Find anomalies in the specified field. SPL example |
series_decompose_anomalies() | KQL example |
where |
Filters search results using eval expressions. Used to compare two different fields. |
where | `T |
Users
| where UserID in ((externaldata (UserID:string) [
@"https://storageaccount.blob.core.windows.net/storagecontainer/users.txt"
h@"?...SAS..." // Secret token to access the blob
])) | ...
Sales
| summarize NumTransactions=count(),
Total=sum(UnitPrice * NumUnits) by Fruit,
StartOfMonth=startofmonth(SellDateTime)
T | summarize count() by price_range=bin(price, 10.0)
sourcetype=MyLogTable type=Event
| transaction ActivityId startswith="Start" endswith="Stop"
| Rename timestamp as StartTime
| Table City, ActivityId, StartTime, Duration
let Events = MyLogTable | where type=="Event";
Events
| where Name == "Start"
| project Name, City, ActivityId, StartTime=timestamp
| join (Events
| where Name == "Stop"
| project StopTime=timestamp, ActivityId)
on ActivityId
| project City, ActivityId, StartTime,
Duration = StopTime – StartTime
Use row_window_session()
to the calculate session start values for a column in a serialized row set.
...| extend SessionStarted = row_window_session(
Timestamp, 1h, 5m, ID != prev(ID))
… | bin span=1m _time
|stats count AS count_i by _time, category
| eventstats sum(count_i) as count_total by _time
Here's an example with the join
statement:
let binSize = 1h;
let detail = SecurityEvent
| summarize detail_count = count() by EventID,
tbin = bin(TimeGenerated, binSize);
let summary = SecurityEvent
| summarize sum_count = count() by
tbin = bin(TimeGenerated, binSize);
detail
| join kind=leftouter (summary) on tbin
| project-away tbin1
Here's an example with the make_list
statement:
let binSize = 1m;
SecurityEvent
| where TimeGenerated >= ago(24h)
| summarize TotalEvents = count() by EventID,
groupBin =bin(TimeGenerated, binSize)
|summarize make_list(EventID), make_list(TotalEvents),
sum(TotalEvents) by groupBin
| mvexpand list_EventID, list_TotalEvents
sourcetype=nasdaq earliest=-10y
| anomalydetection Close _ Price
let LookBackPeriod= 7d;
let disableAccountLogon=SignIn
| where ResultType == "50057"
| where ResultDescription has "account is disabled";
disableAccountLogon
| make-series Trend=count() default=0 on TimeGenerated
in range(startofday(ago(LookBackPeriod)), now(), 1d)
| extend (RSquare,Slope,Variance,RVariance,Interception,
LineFit)=series_fit_line(Trend)
| extend (anomalies,score) =
series_decompose_anomalies(Trend)
SPL command | Description | SPL example | KQL command | KQL example |
---|---|---|---|---|
abs(X) |
Returns the absolute value of X. | abs(number) |
abs() | abs(X) |
case(X,"Y",…) |
Takes pairs of X and Y arguments, where the X arguments are boolean expressions. When evaluated to TRUE , the arguments return the corresponding Y argument. |
SPL example | case | KQL example |
ceil(X) |
Ceiling of a number X. | ceil(1.9) |
ceiling() | ceiling(1.9) |
cidrmatch("X",Y) |
Identifies IP addresses that belong to a particular subnet. | cidrmatch ("123.132.32.0/25",ip) |
• ipv4_is_match() • ipv6_is_match() |
ipv4_is_match('192.168.1.1', '192.168.1.255') == false |
coalesce(X,…) |
Returns the first value that isn't null. | coalesce(null(), "Returned val", null()) |
coalesce() | coalesce(tolong("not a number"), tolong("42"), 33) == 42 |
cos(X) |
Calculates the cosine of X. | n=cos(0) |
cos() | cos(X) |
exact(X) |
Evaluates an expression X using double precision floating point arithmetic. | exact(3.14*num) |
todecimal() | todecimal(3.14*2) |
exp(X) |
Returns eX. | exp(3) |
exp() | exp(3) |
if(X,Y,Z) |
If X evaluates to TRUE , the result is the second argument Y . If X evaluates to FALSE , the result evaluates to the third argument Z . |
if(error==200, "OK", "Error") |
iif() | KQL example |
isbool(X) |
Returns TRUE if X is boolean. |
isbool(field) |
• iif() • gettype |
iif(gettype(X) =="bool","TRUE","FALSE") |
isint(X) |
Returns TRUE if X is an integer. |
isint(field) |
• iif() • gettype |
KQL example |
isnull(X) |
Returns TRUE if X is null. |
isnull(field) |
isnull() | isnull(field) |
isstr(X) |
Returns TRUE if X is a string. |
isstr(field) |
• iif() • gettype |
KQL example |
len(X) |
This function returns the character length of a string X . |
len(field) |
strlen() | strlen(field) |
like(X,"y") |
Returns TRUE if and only if X is like the SQLite pattern in Y . |
like(field, "addr%") |
• has • contains • startswith • matches regex |
KQL example |
log(X,Y) |
Returns the log of the first argument X using the second argument Y as the base. The default value of Y is 10 . |
log(number,2) |
• log • log2 • log10 |
log(X) log2(X) log10(X) |
lower(X) |
Returns the lowercase value of X . |
lower(username) |
tolower | tolower(username) |
ltrim(X,Y) |
Returns X with the characters in parameter Y trimmed from the left side. The default output of Y is spaces and tabs. |
ltrim(" ZZZabcZZ ", " Z") |
trim_start() | trim_start(“ ZZZabcZZ”,” ZZZ”) |
match(X,Y) |
Returns if X matches the regex pattern Y. | match(field, "^\d{1,3}.\d$") |
matches regex | `… |
max(X,…) |
Returns the maximum value in a column. | max(delay, mydelay) |
• max() • arg_max() |
`… |
md5(X) |
Returns the MD5 hash of a string value X . |
md5(field) |
hash_md5 | hash_md5("X") |
min(X,…) |
Returns the minimum value in a column. | min(delay, mydelay) |
• min_of() • min() • arg_min |
KQL example |
mvcount(X) |
Returns the number (total) of X values. |
mvcount(multifield) |
dcount | `… |
mvfilter(X) |
Filters a multi-valued field based on the boolean X expression. |
mvfilter(match(email, "net$")) |
mv-apply | KQL example |
mvindex(X,Y,Z) |
Returns a subset of the multi-valued X argument from a start position (zero-based) Y to Z (optional). |
mvindex( multifield, 2) |
array_slice | array_slice(arr, 1, 2) |
mvjoin(X,Y) |
Given a multi-valued field X and string delimiter Y , and joins the individual values of X using Y . |
mvjoin(address, ";") |
strcat_array | KQL example |
now() |
Returns the current time, represented in Unix time. | now() |
now() | now() now(-2d) |
null() |
Doesn't accept arguments and returns NULL . |
null() |
null | null |
nullif(X,Y) |
Includes two arguments, X and Y , and returns X if the arguments are different. Otherwise, returns NULL . |
nullif(fieldA, fieldB) |
iif | iif(fieldA==fieldB, null, fieldA) |
random() |
Returns a pseudo-random number between 0 to 2147483647 . |
random() |
rand() | rand() |
relative_ time(X,Y) |
Given an epoch time X and relative time specifier Y , returns the epoch time value of Y applied to X . |
relative_time(now(),"-1d@d") |
unix time | KQL example |
replace(X,Y,Z) |
Returns a string formed by substituting string Z for every occurrence of regular expression string Y in string X . |
Returns date with the month and day numbers switched. For example, for the 4/30/2015 input, the output is 30/4/2009 :replace(date, "^(\d{1,2})/ (\d{1,2})/", "\2/\1/") |
replace() | KQL example |
round(X,Y) |
Returns X rounded to the number of decimal places specified by Y . The default is to round to an integer. |
round(3.5) |
round | round(3.5) |
rtrim(X,Y) |
Returns X with the characters of Y trimmed from the right side. If Y isn't specified, spaces and tabs are trimmed. |
rtrim(" ZZZZabcZZ ", " Z") |
trim_end() | trim_end(@"[ Z]+",A) |
searchmatch(X) |
Returns TRUE if the event matches the search string X . |
searchmatch("foo AND bar") |
iif() | iif(field has "X","Yes","No") |
split(X,"Y") |
Returns X as a multi-valued field, split by delimiter Y . |
split(address, ";") |
split() | split(address, ";") |
sqrt(X) |
Returns the square root of X . |
sqrt(9) |
sqrt() | sqrt(9) |
strftime(X,Y) |
Returns the epoch time value X rendered using the format specified by Y . |
strftime(_time, "%H:%M") |
format_datetime() | format_datetime(time,'HH:mm') |
strptime(X,Y) |
Given a time represented by a string X , returns value parsed from format Y . |
strptime(timeStr, "%H:%M") |
format_datetime() | KQL example |
substr(X,Y,Z) |
Returns a substring field X from start position (one-based) Y for Z (optional) characters. |
substr("string", 1, 3) |
substring() | substring("string", 0, 3) |
time() |
Returns the wall-clock time with microsecond resolution. | time() |
format_datetime() | KQL example |
tonumber(X,Y) |
Converts input string X to a number, where Y (optional, default value is 10 ) defines the base of the number to convert to. |
tonumber("0A4",16) |
toint() | toint("123") |
tostring(X,Y) |
Description | SPL example | tostring() | tostring(123) |
typeof(X) |
Returns a string representation of the field type. | typeof(12) |
gettype() | gettype(12) |
urldecode(X) |
Returns the URL X decoded. |
SPL example | url_decode | KQL example |
case(error == 404, "Not found",
error == 500,"Internal Server Error",
error == 200, "OK")
T
| extend Message = case(error == 404, "Not found",
error == 500,"Internal Server Error", "OK")
iif(floor(Timestamp, 1d)==floor(now(), 1d),
"today", "anotherday")
iif(gettype(X) =="long","TRUE","FALSE")
iif(gettype(X) =="string","TRUE","FALSE")
… | where field has "addr"
… | where field contains "addr"
… | where field startswith "addr"
… | where field matches regex "^addr.*"
min_of (expr_1, expr_2 ...)
…|summarize min(expr)
…| summarize arg_min(Price,*) by Product
T | mv-apply Metric to typeof(real) on
(
top 2 by Metric desc
)
strcat_array(dynamic([1, 2, 3]), "->")
let toUnixTime = (dt:datetime)
{
(dt - datetime(1970-01-01))/1s
};
replace( @'^(\d{1,2})/(\d{1,2})/', @'\2/\1/',date)
format_datetime(datetime('2017-08-16 11:25:10'),
'HH:mm')
format_datetime(datetime(2015-12-14 02:03:04),
'h:m:s')
Returns a field value of X
as a string.
- If the value of
X
is a number,X
is reformatted to a string value. - If
X
is a boolean value,X
is reformatted toTRUE
orFALSE
. - If
X
is a number, the second argumentY
is optional and can either behex
(convertsX
to a hexadecimal),commas
(formatsX
with commas and two decimal places), orduration
(convertsX
from a time format in seconds to a readable time format:HH:MM:SS
).
This example returns:
foo=615 and foo2=00:10:15:
… | eval foo=615 | eval foo2 = tostring(
foo, "duration")
urldecode("http%3A%2F%2Fwww.splunk.com%2Fdownload%3Fr%3Dheader")
SPL command | Description | KQL command | KQL example |
---|---|---|---|
avg(X) |
Returns the average of the values of field X . |
avg() | avg(X) |
count(X) |
Returns the number of occurrences of the field X . To indicate a specific field value to match, format X as eval(field="value") . |
count() | summarize count() |
dc(X) |
Returns the count of distinct values of the field X . |
dcount() | …| summarize countries=dcount(country) by continent |
earliest(X) |
Returns the chronologically earliest seen value of X . |
arg_min() | … | summarize arg_min(TimeGenerated, *) by X |
latest(X) |
Returns the chronologically latest seen value of X . |
arg_max() | … | summarize arg_max(TimeGenerated, *) by X |
max(X) |
Returns the maximum value of the field X . If the values of X are non-numeric, the maximum value is found via alphabetical ordering. |
max() | …| summarize max(X) |
median(X) |
Returns the middle-most value of the field X . |
percentile() | …| summarize percentile(X, 50) |
min(X) |
Returns the minimum value of the field X . If the values of X are non-numeric, the minimum value is found via alphabetical ordering. |
min() | …| summarize min(X) |
mode(X) |
Returns the most frequent value of the field X . |
top-hitters() | …| top-hitters 1 of Y by X |
perc(Y) |
Returns the percentile X value of the field Y . For example, perc5(total) returns the fifth percentile value of a field total . |
percentile() | …| summarize percentile(Y, 5) |
range(X) |
Returns the difference between the maximum and minimum values of the field X . |
range() | range(1, 3) |
stdev(X) |
Returns the sample standard deviation of the field X . |
stdev | stdev() |
stdevp(X) |
Returns the population standard deviation of the field X . |
stdevp() | stdevp() |
sum(X) |
Returns the sum of the values of the field X . |
sum() | sum(X) |
sumsq(X) |
Returns the sum of the squares of the values of the field X . |
||
values(X) |
Returns the list of all distinct values of the field X as a multi-value entry. The order of the values is alphabetical. |
make_set() | …| summarize r = make_set(X) |
var(X) |
Returns the sample variance of the field X . |
variance | variance(X) |
In this article, you learned how to map your migration rules from Splunk to Microsoft Sentinel.
[!div class="nextstepaction"] Migrate your SOAR automation