Skip to content

feat: add security.txt detection flag #2472

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
iacker wants to merge 1 commit into
base: dev
Choose a base branch
from
+117 −0
Open

feat: add security.txt detection flag #2472

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions runner/options.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,7 @@ type Options struct {
OutputIP bool
OutputCName bool
ExtractFqdn bool
SecurityTxt bool
Unsafe bool
Debug bool
DebugRequests bool
Expand Down Expand Up @@ -417,6 +418,7 @@ func ParseOptions() *Options {
flagSet.BoolVar(&options.OutputIP, "ip", false, "display host ip"),
flagSet.BoolVar(&options.OutputCName, "cname", false, "display host cname"),
flagSet.BoolVarP(&options.ExtractFqdn, "efqdn", "extract-fqdn", false, "get domain and subdomains from response body and header in jsonl/csv output"),
flagSet.BoolVar(&options.SecurityTxt, "security-txt", false, "detect a valid security.txt file on standard paths"),
flagSet.BoolVar(&options.Asn, "asn", false, "display host asn information"),
flagSet.DynamicVar(&options.OutputCDN, "cdn", "true", "display cdn/waf in use"),
flagSet.BoolVar(&options.Probe, "probe", false, "display probe status"),
Expand Down
57 changes: 57 additions & 0 deletions runner/runner.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -330,6 +330,15 @@ func New(options *Options) (*Runner, error) {
scanopts.VHostInput = options.VHostInput
scanopts.OutputContentType = options.OutputContentType
scanopts.RequestBody = options.RequestBody
if options.SecurityTxt {
options.OutputMatchString = append(options.OutputMatchString, "Contact:")
options.OutputMatchStatusCode = appendCommaSeparatedValue(options.OutputMatchStatusCode, "200")
options.RequestURIs = appendCommaSeparatedValue(options.RequestURIs, "/.well-known/security.txt,/security.txt")
options.JSONOutput = true
}
if options.RequestURIs != "" {
options.requestURIs = normalizeRequestURIs(options.RequestURIs)
}
Comment on lines +333 to +341
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Critical: OutputMatchStatusCode modification occurs after parsing, making status code filter ineffective.

When --security-txt is enabled, this code appends "200" to options.OutputMatchStatusCode. However, ValidateOptions() (called before New() in ParseOptions()) has already parsed OutputMatchStatusCode into the matchStatusCode slice used for filtering at line 1204.

As a result, the 200 status code match won't be enforced when using --security-txt alone, potentially returning non-200 responses.

🐛 Proposed fix: Parse the status code after modification 
 if options.SecurityTxt {
     options.OutputMatchString = append(options.OutputMatchString, "Contact:")
     options.OutputMatchStatusCode = appendCommaSeparatedValue(options.OutputMatchStatusCode, "200")
     options.RequestURIs = appendCommaSeparatedValue(options.RequestURIs, "/.well-known/security.txt,/security.txt")
     options.JSONOutput = true
+    // Re-parse matchStatusCode since we modified OutputMatchStatusCode after initial parsing
+    if parsed, err := stringz.StringToSliceInt(options.OutputMatchStatusCode); err == nil {
+        options.matchStatusCode = parsed
+    }
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if options.SecurityTxt {
options.OutputMatchString = append(options.OutputMatchString, "Contact:")
options.OutputMatchStatusCode = appendCommaSeparatedValue(options.OutputMatchStatusCode, "200")
options.RequestURIs = appendCommaSeparatedValue(options.RequestURIs, "/.well-known/security.txt,/security.txt")
options.JSONOutput = true
}
if options.RequestURIs != "" {
options.requestURIs = normalizeRequestURIs(options.RequestURIs)
}
if options.SecurityTxt {
options.OutputMatchString = append(options.OutputMatchString, "Contact:")
options.OutputMatchStatusCode = appendCommaSeparatedValue(options.OutputMatchStatusCode, "200")
options.RequestURIs = appendCommaSeparatedValue(options.RequestURIs, "/.well-known/security.txt,/security.txt")
options.JSONOutput = true
// Re-parse matchStatusCode since we modified OutputMatchStatusCode after initial parsing
if parsed, err := stringz.StringToSliceInt(options.OutputMatchStatusCode); err == nil {
options.matchStatusCode = parsed
}
}
if options.RequestURIs != "" {
options.requestURIs = normalizeRequestURIs(options.RequestURIs)
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@runner/runner.go` around lines 333 - 341, The code appends "200" to
options.OutputMatchStatusCode when options.SecurityTxt is true, but parsing into
the internal matchStatusCode slice happens earlier in
ValidateOptions()/ParseOptions(), so the appended 200 is ignored; to fix, ensure
the status-code parsing runs after you modify OutputMatchStatusCode — either
move the append logic for options.OutputMatchStatusCode (and any related
options.RequestURIs changes) to before ValidateOptions()/New() in
ParseOptions(), or call the same parsing routine that produces matchStatusCode
again after the append so matchStatusCode includes "200" (refer to
options.SecurityTxt, options.OutputMatchStatusCode, ValidateOptions(),
ParseOptions(), New(), and the matchStatusCode parsing logic).

scanopts.Unsafe = options.Unsafe
scanopts.Pipeline = options.Pipeline
scanopts.HTTP2Probe = options.HTTP2Probe
Expand Down Expand Up @@ -2632,6 +2641,11 @@ retry:
}
}

securityTxt := false
if r.options.SecurityTxt {
securityTxt = isSecurityTxt(resp)
}

result := Result{
Timestamp: time.Now(),
Request: request,
Expand All @@ -2650,6 +2664,7 @@ retry:
StatusCode: resp.StatusCode,
Location: resp.GetHeaderPart("Location", ";"),
ContentType: resp.GetHeaderPart("Content-Type", ";"),
SecurityTxt: securityTxt,
Title: title,
str: builder.String(),
VHost: isvhost,
Expand Down Expand Up @@ -2973,6 +2988,48 @@ func (r Result) CSVRow(scanopts *ScanOptions) string { //nolint
return res
}

func appendCommaSeparatedValue(current string, values string) string {
current = strings.TrimSpace(current)
values = strings.TrimSpace(values)
if current == "" {
return values
}
if values == "" {
return current
}
return current + "," + values
}

func normalizeRequestURIs(requestURIs string) []string {
items := stringsutil.SplitAny(requestURIs, ",")
cleaned := make([]string, 0, len(items))
seen := make(map[string]struct{}, len(items))
for _, item := range items {
item = strings.TrimSpace(item)
if item == "" {
continue
}
if _, ok := seen[item]; ok {
continue
}
seen[item] = struct{}{}
cleaned = append(cleaned, item)
}
return cleaned
}

func isSecurityTxt(resp *httpx.Response) bool {
if resp == nil || resp.StatusCode != 200 {
return false
}
contentType := strings.ToLower(resp.GetHeaderPart("Content-Type", ";"))
if contentType != "" && !strings.Contains(contentType, "text/plain") {
return false
}
body := string(resp.RawData)
return strings.Contains(body, "Contact:")
}
Comment on lines +3021 to +3031
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🌐 Web query:

RFC 9116 security.txt field name case sensitivity

💡 Result:

RFC 9116 specifies that security.txt field names are case-insensitive. It defines the field “name” as a field-name per RFC 5322 and then explicitly states: “Field names are case insensitive (as per Section 2.3 of RFC 5234).” (datatracker.ietf.org)

Citations:

Case-sensitive "Contact:" check may miss valid security.txt files.

Per RFC 9116, field names in security.txt are case-insensitive. The current check strings.Contains(body, "Contact:") will miss valid files using lowercase contact: or mixed case variants.

Suggested fix 
 func isSecurityTxt(resp *httpx.Response) bool {
     if resp == nil || resp.StatusCode != 200 {
         return false
     }
     contentType := strings.ToLower(resp.GetHeaderPart("Content-Type", ";"))
     if contentType != "" && !strings.Contains(contentType, "text/plain") {
         return false
     }
-    body := string(resp.RawData)
-    return strings.Contains(body, "Contact:")
+    body := strings.ToLower(string(resp.RawData))
+    return strings.Contains(body, "contact:")
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func isSecurityTxt(resp *httpx.Response) bool {
if resp == nil || resp.StatusCode != 200 {
return false
}
contentType := strings.ToLower(resp.GetHeaderPart("Content-Type", ";"))
if contentType != "" && !strings.Contains(contentType, "text/plain") {
return false
}
body := string(resp.RawData)
return strings.Contains(body, "Contact:")
}
func isSecurityTxt(resp *httpx.Response) bool {
if resp == nil || resp.StatusCode != 200 {
return false
}
contentType := strings.ToLower(resp.GetHeaderPart("Content-Type", ";"))
if contentType != "" && !strings.Contains(contentType, "text/plain") {
return false
}
body := strings.ToLower(string(resp.RawData))
return strings.Contains(body, "contact:")
}
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@runner/runner.go` around lines 3021 - 3031, The isSecurityTxt function only
checks for the exact case "Contact:" which can miss valid security.txt field
names; update the check in isSecurityTxt to perform a case-insensitive search
for the "contact:" field (e.g., normalize resp.RawData to lowercase and check
for "contact:" or use a case-insensitive regex) so that variants like "contact:"
or "CONTACT:" are detected while keeping the existing Content-Type and status
checks intact.


func (r *Runner) skipCDNPort(host string, port string) bool {
// if the option is not enabled we don't skip
if !r.scanopts.ExcludeCDN {
Expand Down
57 changes: 57 additions & 0 deletions runner/runner_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,63 @@ func TestRunner_urlWithComma_targets(t *testing.T) {
require.ElementsMatch(t, expected, got, "could not expected output")
}

func TestNormalizeRequestURIs(t *testing.T) {
got := normalizeRequestURIs("/.well-known/security.txt, /security.txt, /.well-known/security.txt,,")
require.Equal(t, []string{"/.well-known/security.txt", "/security.txt"}, got)
}

func TestAppendCommaSeparatedValue(t *testing.T) {
require.Equal(t, "200", appendCommaSeparatedValue("", "200"))
require.Equal(t, "200,302", appendCommaSeparatedValue("200", "302"))
require.Equal(t, "200", appendCommaSeparatedValue("200", ""))
}

func TestIsSecurityTxt(t *testing.T) {
t.Run("valid security txt", func(t *testing.T) {
resp := &httpx.Response{
StatusCode: 200,
Headers: map[string][]string{
"Content-Type": {"text/plain; charset=utf-8"},
},
RawData: []byte("Contact: mailto:security@example.com\nExpires: 2027-01-01T00:00:00.000Z\n"),
}
require.True(t, isSecurityTxt(resp))
})

t.Run("soft 404 html", func(t *testing.T) {
resp := &httpx.Response{
StatusCode: 200,
Headers: map[string][]string{
"Content-Type": {"text/html; charset=utf-8"},
},
RawData: []byte("<html><body>Contact: support@example.com</body></html>"),
}
require.False(t, isSecurityTxt(resp))
})

t.Run("missing contact field", func(t *testing.T) {
resp := &httpx.Response{
StatusCode: 200,
Headers: map[string][]string{
"Content-Type": {"text/plain"},
},
RawData: []byte("Expires: 2027-01-01T00:00:00.000Z\n"),
}
require.False(t, isSecurityTxt(resp))
})

t.Run("wrong status", func(t *testing.T) {
resp := &httpx.Response{
StatusCode: 404,
Headers: map[string][]string{
"Content-Type": {"text/plain"},
},
RawData: []byte("Contact: mailto:security@example.com\n"),
}
require.False(t, isSecurityTxt(resp))
})
}

func TestRunner_CSVRow(t *testing.T) {
// Create a result with fields that would be vulnerable to CSV injection
result := Result{
Expand Down
1 change: 1 addition & 0 deletions runner/types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ type Result struct {
ResponseBody string `json:"body,omitempty" csv:"-" md:"-" mapstructure:"body"`
BodyPreview string `json:"body_preview,omitempty" csv:"body_preview" md:"body_preview" mapstructure:"body_preview"`
ContentType string `json:"content_type,omitempty" csv:"content_type" md:"content_type" mapstructure:"content_type"`
SecurityTxt bool `json:"security_txt,omitempty" csv:"security_txt" md:"security_txt" mapstructure:"security_txt"`
Method string `json:"method,omitempty" csv:"method" md:"method" mapstructure:"method"`
Host string `json:"host,omitempty" csv:"host" md:"host" mapstructure:"host"`
HostIP string `json:"host_ip,omitempty" csv:"host_ip" md:"host_ip" mapstructure:"host_ip"`
Expand Down