Firefox 3.6.13: damn you, corner cases

Today's release of Firefox 3.6.13 fixes bug 602780 (CVE-2010-3774) - an interesting problem I originally reported to Mozilla in October. It's a fun one, so here's a quick recap of what it is about.

As you may recall, one of the more significant shortcomings of the same-origin policy is that it does not give any guidance on handling documents with no inherent origin associated - that is, it fails to account for all the content coming from about:, data:, file:, and similar pseudo-URLs. Consequently, early implementations of this security model simply allowed all origin-less frames or windows created by one site to be accessed freely across domains - an empty host name string equals any other empty host name string, after all. As can be expected, this approach proved to be a bad idea - and since then, quite a few necessary improvements have been made. Today, every pseudo-URL should either inherit its origin from the parent document, or be assigned a completely unique one.

Alas, in Firefox, some of this logic would not work as expected in a handful of corner cases; this problem probably traced back to a minor code refactoring somewhere in 2008. The vulnerability would not trigger with about: or data: subframes on legitimate sites - and therefore, would not affect normal browsing - but with some minimal effort, it could be leveraged by malicious sites to access and modify the contents of internal browser pages, such as about:config, about:neterror, and so forth.

The consequences of access to about:config can be disastrous, but in this case, are somewhat mitigated by the fact that under standard operating conditions, random Internet-originating content can't open that location directly. In other words, the vulnerability could not be exploited without soliciting a degree of user interaction.

The story with pages such as about:neterror is a bit more interesting, though: in modern versions of Firefox, these documents are shown in place of the old-fashioned modal dialogs to indicate navigation errors. Crucially, when this happens, the content is displayed with the intended destination URL, rather than the true origin (about:...), shown in the address bar. With this simple trick in mind, the attacker can inject his spoofed content into a window where the address bar incorrectly points to an unrelated domain of his choice. Whoops!

If you haven't upgraded to 3.6.13 yet, check out this bombastic demo to see the attack in action.

Read More

Weekend distractions: DHS threat level indicator

Cool vulnerability in Firefox coming up next week. In the meantime, enjoy this project of mine:

• DHS tribute threat level indicator
  

Yup. The name says it all.

Read More

Understanding and using skipfish

Skipfish, my open source web application security scanner, is now about eight months old - and, over the course of over 70 releases, has undergone a number of substantial changes. While I maintain detailed documentation and a short troubleshooting guide, it seems appropriate to share some additional hints on how to get the most out of this tool, should you be inclined to try it out.

A word on design goals (and what they really mean)

While skipfish tries to scratch quite a few itches, its primary goals - and the areas where it hopefully stands out - are:

• Raw speed: I am constantly frustrated by the performance and memory footprint of many of the open source and commercial scanners and brute-force tools I had to work with. Skipfish tries hard to improve this - and to my knowledge, has by far the fastest HTTP and content analysis engine out there.
  

  
  This does not mean that skipfish scans always take the least amount of time, compared to other tools; it simply means that I can cram a whole lot more functionality, and get much better coverage, without making the assessment unreasonably long.
  

  
  In cases where the server is the bottleneck, this can obviously backfire; but when dealing with slow targets, you can configure the scanner to get a reduced coverage - roughly comparable to a more traditional tool.
  

  
  

• Unique brute-force capabilities: the performance of the scanner allowed me to incorporate an extensive ${keyword}.${extension} brute-force functionality similar to that of DirBuster - coupled with highly customized, hand-picked dictionaries, and a unique auto-learning feature that builds an adaptive, target-specific dictionary based on site content analysis.
  

  
  Most other scanners simply can't afford to let you do this in any meaningful way - and when they do, the dictionaries you can use with them are much less sophisticated, and far more request-intensive. I consider this functionality to be one of the more important assets of the tool - but you are certainly not forced to use it where impractical. The brute-force testing features are completely optional, and can be turned off to improve scan times as much as 500-fold.
  

  
  

• High quality security checks: most scanners employ fairly naive security logic - for example, to test cross-site scripting, they may attempt injecting <script>alert(1)</script>; to detect directory traversal, they may try ../../../../../etc/passwd; and to test SQL injection, they may attempt supplying technology-specific code and look for equally technology-specific output strings.
  

  
  Needless to say, all these checks have many painfully simple failure modes: the XSS check will result in a false negative when the input is partly escaped - or appears inside a HTML comment; traversal checks will fail if the application always appends a fixed extension to the input string, or is running within a chroot() jail; and SQL injection logic will break when dealing with an unfamiliar backend or an uncommon application framework.
  

  
  To that effect, skipfish puts emphasis on well-crafted probes, and on testing for behavioral patterns, rather than signatures. For example, when testing for string-based SQL injection, we compare the results of passing '"original_value, \'\"original_value, and \\'\\"original_value. When the first response is similar to the third one, but different from from the second one - we can, with a pretty high confidence, say that there is an underlying query injection vulnerability (even if query results can't be observed directly). Interestingly, this check is versatile enough to do a pretty good job detecting eval()-related vulnerabilities in PHP, and injection bugs in many other non-SQL query languages.
  

  
  Similarly, when probing for file inclusion, the scanner will try to compare the output of original_value, ./original_value, ../original_value, and .../original_value; if the first two cases result in similar output, and the two remaining ones result in a different outcome, we are probably dealing with a traversal problem. The redundancy helps rule out differences that can be attributed to input validation - and hey, that check also triggers on many remote file inclusion vectors in PHP.
  

  
  For XSS, skipfish does depend on content analysis - but instead of the standard practice of throwing the entire XSS cheatsheet at the target, it injects a complex string that is guaranteed to break out of many different parsing modes (and much less likely to fail with crude XSS filters in place): -->">'>'", followed by a uniquely numbered tag. The unique identifier enables stored XSS detection later on, and is also interpreted in a special way inside <script> or <style> blocks.
  

  
  There are many other design decisions along these lines; I believe they have a profound impact on the ability to detect real-world security problems - although paradoxically, they make the scanner perform poorly with simulated vulnerabilities in demo sites.
  

  
  

• Coverage of more nuanced problems: most web application assessment tools simply pay no attention to the security risks caused by subtle MIME type or character set mismatches - and the awareness that these problems may lead to exploitable XSS vectors is very low within the security community.
  

  
  Skipfish makes a point of noticing these and many other significant security issues usually neglected by other tools - such as caching intent mismatches, mixed content issues, XSSI, third-party scripts, cross-site request forgery, and so forth.
  

  
  Quite a few people complained about "pointless" or "odd" warnings for problems thought to be non-security issues. Rest assured, most of these cases could be shown to be exploitable. When in doubt, don't hesitate to ping me for a second opinion!
  

  
  

• Adaptive scanning for real-world applications: many scanners don't handle complex, mixed technology sites particularly well; a common headache is dealing with closely related requests being handled by different application backends with different URL semantics, error messages, or even case-sensitivity. Another hard problem is recognizing obscure 404 behaviors, unusual parameter passing conventions, redirection patterns, content duplication, and so forth. All this often requires repeated, painstaking configuration tweaks.
  

  
  While skipfish certainly is not perfect - no scanner can be - the code is designed to be able to cope with these scenarios exceptionally well. This is achieved chiefly by not relying on directory, file, or error message signatures - and instead, carrying out adaptive probes for every new fuzzed location; and quickly recognizing crawl tree branches that look very much alike. While heuristics can fail in unexpected ways, I think it's of immense value.
  
  

  
  

• Sleek reports with very little noise: skipfish generally does not complain about highly non-specific "vulnerabilities" commonly reported by other scanners; for example, it does not pay special attention to every non-httponly cookie, to every password form with autocomplete enabled, or to framework version or system path disclosure patterns on various pages. This means that in practice, auditors will see fewer issues in a skipfish report than in the output of most other assessment tools - and this is not a bug.
  

  
  Rest assured, the interactive report produced after a scan includes summary sections where the auditor can review all password forms, cookies, and so forth if necessary - but the assumption is that human evaluation can't and should not be substituted here.
  

While skipfish certainly isn't perfect, these are the core properties I care about - and try to continually improve.

The most important setting: dictionary modes

The single most misunderstood - and important - feature of skipfish is its directory management model. Quite simply, getting this part wrong can easily ruin your scans.

I encourage you to have a look at the recently revamped dictionaries/README-FIRST file, which explains the basics of dictionary management in greater detail; but at the very minimum, you should be aware of the following choice:

• Absolutely no brute-force: in this mode, skipfish performs an orderly crawl of the target site, and behaves similarly to other basic scanners. The mode is not recommended in most cases due to limited coverage - resources such as /admin/ or /index.php.old may not be discovered - but is blazing fast. To use it, try:
  

  
  ./skipfish -W /dev/null -LV [...other options...]
  

  
  
  
• Lightweight brute-force: in this mode, the scanner will only try fuzzing the file name (/admin/), or the extension (/index.php.old), but never both at the same time (/backup.tgz will typically not be hit). The cost of doing so is about 1,700 requests per fuzzed location. To use this mode, try:
  

  
  cp dictionaries/complete.wl dictionary.wl
  ./skipfish -W dictionary.wl -Y [...other options...]
  

  
  This mode is the preferred way of limiting scan time where fully-fledged brute-force testing is not feasible.
  

  
  

• Normal dictionary brute-force: in this mode, the scanner will test all the possible file name and extension pairs (i.e., /backup.tgz will be discovered, too). The mode is significantly slower, but offers superior coverage - and should be your default pick in most cases. To enable it, try:
  

  
  cp dictionaries/minimal.wl dictionary.wl
  ./skipfish -W dictionary.wl [...other options...]
  

  
  The cost of this mode is about 50,000 requests per fuzzed location. You can replace minimal.wl with medium.wl or complete.wl for even better coverage, but at the expense of a 2x to 3x increase in scan time; see dictionaries/README-FIRST for an explanation of the difference between these files.
  
  

Other options you need to know about

Skipfish requires relatively little configuration, but rest assured, is not a point-and-click tool. You should definitely review the documentation to understand the operation of rudimentary options such as -C (use cookie authentication), -I (only crawl matching URLs), -X (exclude matching URLs), -D (define domain scope), -m (limit simultaneous connections), and so forth.

Here is a list of some of the more useful but under-appreciated options that you should consider using in your work:

• Limit crawl tree fanout: options -c (immediate children limit) and -x (total descendant limit) allow you to fine-tune scan coverage for very large sites where -I and -X options are impractical to use. Non-deterministic crawl probability setting (-p) may also be helpful there.
  

  
  

• More SSL checks: some sites care about getting SSL right; specify -M to warn about dangerous mixed content scenarios and insecure password forms.
  

  
  

• Spoof another browser: use -b to convincingly pretend to be MSIE, Firefox, or an iPhone. This option does not merely change the User-Agent string, but also ensures that other headers have the right syntax and ordering.
  

  
  

• Do not accept new cookies: on cookie-authenticated sites that do not maintain session state on server side, accidental logout can be prevented without the need to carefully specify -X locations: adding -N simply instructs skipfish to ignore all attempts to delete or modify -C cookies.
  

  
  

• Trust another domain: skipfish complains about dangerous script inclusion and content embedding from third-party domains, and can optionally warn you about outgoing links. To minimize noise, use -B to identify any domains you trust, and therefore, want to exclude from these checks.
  

  
  

• Reduce memory footprint: to generate scan reports, the scanner keeps samples of all retrieved documents in memory. On some large, multimedia-heavy sites, this may consume a lot of RAM. In these cases, -e may be used to purge binary (non-ASCII) content without impacting report quality appreciably.
  

  
  

• Be dumb: by specifying -O, skipfish can be instructed not to analyze returned HTML to extract links - turning it into a fast, purely brute-force tool.
  

Troubleshooting

Skipfish has bugs, there are features yet to be implemented, and web frameworks it hasn't encountered yet. When you run into any trouble, please check out this doc, but also do not hesitate to ping me directly. Your feedback is the only way this tool can be improved.

Read More

Dyr suksess

Takket være en ”overivrig” Riskrevisjonen vet vi nå mer om hvilke bonusavtaler de eksterne forvalterne av Oljefondet har. NBIM ,som forvalter oljefondet for Norges Bank, må gjerne korrigere meg, men det ser ut til at en svensk forvalter har fått en bonus på 40 % av en meravkastning på nær halvannen milliard kroner.
Som kjent påtar ingen av Oljefondets forvaltere seg ansvar for å betale tilbake penger som de taper. Det verste som kan skje er at de mister oppdrag i fremtiden. Dette gir en åpenbar motivasjon til å øke risikoen på en måte som ikke lett kan avdekkes av fondet. Økningen i risiko har til og med en konkret verdi i kroner og øre. Slike kontrakter kalles nemlig opsjoner og vi har ganske eksakte metoder for å verdsette dem, som vist i figuren.

Dersom forvalteren for eksempel innretter seg slik at en kan forvente tap eller gevinst i forhold til referanseporteføljen i størrelsesorden 20%, så ser vi av figuren at verdien på kontrakten vil være litt i overkant av 500 millioner. Forvalteren kan i prinsippet selv velge hvor på den røde linjen han ønsker å være. Det virker derfor ikke usannsynlig at noen forvaltere vil forsøke å maksimere verdien på kontrakten ved å ta mest mulig risiko.

Enda mer interessant er det kanskje at forvalteren ikke trenger å vente på å innkassere denne gevinsten. Forvalteren kan nemlig hypotetisk sett selge opsjoner i markedet og umiddelbart motta et beløp tilsvarende verdien av kontrakten. Nå er det ingen holdepunkter for at slik har skjedd, men det er jo et tankekors at dette er praktisk fullt gjennomførbart.

NBIM har utvilsomt omfattende systemer for å kontrollere risikoen som de eksterne forvalterne påfører fondet. Problemet er at denne kontrollen er langt fra perfekt og må skje i ettertid basert på den informasjon som tilflyter NBIM. Den katastrofale erfaringen fra obligasjonsinvesteringene før finanskrisen tilsier at kontrollen er utilstrekkelig. Det kan godt hende at forvalterne da spekulerte i å ta en type risiko som NBIM ikke overvåket nettopp for å innkassere opsjonsgevinster som nevnt.

Nå er ikke opsjonen som utstedes helt kostnadsfri. Forvalteren kan miste oppdraget. Forvaltere taper imidlertid i gjennomsnitt mot markedet på grunn av kostnader. Verdien av å drive innenfor markedets risikorammer er derfor veldig liten. Forvaltere som følger spillereglene kan rett og slett ikke regne med å få noen særlig bonus. Det bør derfor ikke overraske om fristelsen til å laste opp med risiko av og til blir for stor.

I mange tilfeller vil imidlertid tap hos en forvalter dekkes av gevinst hos en annen. Hovedproblemet er derfor ikke risikoen, men de kostnadsdrivende bonusene. Siden fondet ikke får tilbakebetalt når en forvalter taper, så må fondet sette sin lit til at forvalteren med 40% bonus slår markedet mer enn 62,5% av tiden. Hvis ikke vil fondet systematisk betale ut store beløp hver gang en forvalter er heldig, og sitte igjen med regningen når det går skeis. En må derfor kunne stille spørsmål ved om ikke bonuser av denne typen forutsetter et i overkant optimistisk syn på hva eksterne forvaltere kan få til.

Ovennevnte problem kan til en viss grad avhjelpes ved å sette bonustak. NBIM har imidlertid forsvart avtalen uten å ta selvkritikk og åpnet for at de kan komme til å inngå lignende avtaler i fremtiden. Vi får håpe de har gode argumenter for det.
Et argument som NBIM har fremført er at disse kontraktene er nødvendig for at forvalterne skal ta oppdraget. Fondet har tidligere anført sin evne til å oppnå gode avtaler med eksterne forvaltere som ett av sine komparative fortrinn og et argument for aktiv forvaltning. Det virker plutselig ikke så troverdig lenger.

Aktiv forvaltning har økt avkastningen på fondet med ca 0,1% per år i forhold til indeksforvaltning. NBIM påstår at bidraget egentlig er større ved å vise til store indirekte kostnader ved passiv forvaltning, men fondets eget interne indeksfond ikke har slike kostnader. Denne lille gevinsten kan videre forklares ut fra velkjente strategier som en ikke trenger aktiv forvaltning for å følge. Det ser altså ikke ut som en får særlig mye igjen for den aktive forvaltningen. Er det da verdt det?

 

Se NBIMs svar her

Read More

Uten argumenter?

Det har ikke vært noen respons på mitt innlegg i DN om en såkalt tobinskatt på aksjer, hverken fra tilhengerne i LO eller Attac. Argumentasjonen i innlegget er solid, men jeg hadde likevel ventet noe motbør. En må vel da konkludere med at tilhengerne ikke finner vektige argumenter utover de gamle som jeg har tilbakevist og at det ikke finnes gode grunner for å innføre en slik skatt?

Read More

These are not the events you are looking for

Yeah, so this probably should not be possible.

The underlying problem is pretty cute: most browsers can be programatically prevented from dequeuing and processing UI events delivered by the operating system; canonical examples involve using busy JavaScript loops, blocking XMLHttpRequest calls, and particularly complex HTML or XML documents.

Upon leaving this state, the queued events may not be properly purged, and may end up getting delivered to an incorrect and unexpected context - possibly carrying out an undesirable action in another domain, or interacting with browser chrome.

I filed bug 608899 for this particular demo in Firefox - but given the general, cross-browser state of disrepair when it comes to UI timing and related attacks, I am not getting my hopes up.

Read More

HTTP cookies, or how not to design protocols

For as long as I remember, HTTP cookies have been vilified as a grave threat to the privacy of online browsing; wrongly so. That said, the mechanism itself is a very interesting cautionary tale for security engineers - and that will be the theme of today's feature.

Cookies were devised by Lou Montulli, a Netscape engineer, somewhere in 1994. Lou outlined his original design in a minimalistic, four-page proposal posted on netscape.com; based on that specification, the implementation shipped in their browser several months later - and other vendors were quick to follow.

It wasn't until 1997 that the first reasonably detailed specification of the mechanism has been attempted: RFC 2109. The document captured some of the status quo - but confusingly, also tried to tweak the design, an effort that proved to be completely unsuccessful; for example, contrary to what is implied by this RFC, most browsers do not support multiple comma-delimited NAME=VALUE pairs in a single Set-Cookie header; do not recognize quoted-string cookie values; and do not use max-age to determine cookie lifetime.

Three years later, another, somewhat better structured effort to redesign cookies - RFC 2965 - proved to be equally futile. Meanwhile, browser vendors tweaked or extended the scheme in their own ways: for example, around 2002, Microsoft unilaterally proposed httponly cookies as a security mechanism to slightly mitigate the impact of cross-site scripting flaws - a concept quickly, if prematurely, embraced by the security community.

All these moves led to a very interesting situation: there is simply no accurate, official account of cookie behavior in modern browsers; the two relevant RFCs, often cited by people arguing on the Internet, are completely out of touch with reality. This forces developers to discover compatible behaviors by trial and error - and makes it an exciting gamble to build security systems around cookies in the first place.

In any case - well-documented or not, cookies emerged as the canonical solution to an increasingly pressing problem of session management; and as web applications have grown more complex and more sensitive, the humble cookie caught the world by storm. With it, came a flurry of fascinating security flaws.

They have Internet over there, too?

Perhaps the most striking issue - and an early sign of trouble - is the problem of domain scoping.

Unlike the more pragmatic approach employed for JavaScript DOM access, cookies can be set for any domain of which the setter is a member - say, foo.example.com is meant to be able to set a cookie for *.example.com. On the other hand, allowing example1.com to set cookies for example2.com is clearly undesirable, as it allows a variety of sneaky attacks: denial of service at best, and altering site preferences, modifying carts, or stealing personal data at worst.

To that effect, the RFC provided this elegant but blissfully naive advice:

"Only hosts within the specified domain can set a cookie for a domain and domains must have at least two (2) or three (3) periods in them to prevent domains of the form: ".com", ".edu", and "va.us". Any domain that fails within one of the seven special top level domains listed below only require two periods. Any other domain requires at least three. The seven special top level domains are: "COM", "EDU", "NET", "ORG", "GOV", "MIL", and "INT".

Regrettably, there are at least three glaring problems with this scheme - two of which should have been obvious right away:

1. Some country-level registrars indeed mirror the top-level hierarchy (e.g. example.co.uk), in which case the three-period rule makes sense; but many others allow direct registrations (e.g., example.fr), or permit both approaches to coexist (say, example.jp and example.co.jp). In the end, the three-period rule managed to break cookies in a significant number of ccTLDs - and consequently, most implementations (Netscape included) largely disregarded the advice. Yup, that's right - as a result, you could set cookies for *.com.pl.
   

   
   

2. The RFC missed the fact that websites are reachable by means other than their canonical DNS names; in particular, the rule permitted a website at http://1.2.3.4/ to set cookies for *.3.4, or a website at http://example.com.pl./ to set a cookie for *.com.pl.
   

   
   

3. To add insult to injury, Internet Assigned Numbers Authority eventually decided to roll out a wide range of new top-level domains, such as .biz, .info, or .jobs - and is now attempting to allow arbitrary gTLD registrations. This last step promises to be a yet another nail to the coffin of sane cookie management implementations.
   

Net effect? All mainstream browsers had a history of embarrassing bugs in this area - and now ship with a giant, hairy, and frequently-updated lists of real-world "public suffix" domains for which cookies should not be set - as well as an array of checks to exclude non-FQDN, IPs, and pathological DNS notations of all sorts.

8K ought to be enough for for anybody

To make denial-of-service attacks a bit harder, it is well-understood that most web servers limit the size of a request they are willing to process; these limits are very modest - for example, Apache rejects request headers over 8 kB, while IIS draws the line at 16 kB. This is perfectly fine under normal operating conditions - but can be easily exceeded when the browser is attempting to construct a request with a lot of previously set cookies attached.

The specification neglected this possibility, offered no warning to implementators, and proposed no discovery and resolution algorithm. In fact, it mandated minimal jar size requirements well in excess of the limits enforced by HTTP servers:

"In general, user agents' cookie support should have no fixed limits. They should strive to store as many frequently-used cookies as possible. Furthermore, general-use user agents should provide each of the following minimum capabilities [...]:

* at least 300 cookies

* at least 4096 bytes per cookie (as measured by the size of the characters that comprise the cookie non-terminal in the syntax description of the Set-Cookie header)

* at least 20 cookies per unique host or domain name"

As should be apparent, the suggested minimum - 20 cookies of 4096 bytes each - allows HTTP request headers to balloon up to the 80 kB boundary.

Does this matter from the security perspective? At first sight, no - but this is only until you realize that there are quite a few popular sites that rely on user-name.example.com content compartmentalization; and that any malicious user can set top-level cookies to prevent the visitor from ever being able to access any *.example.com site again.

The only recourse domain owners have in this case is to request their site to be added to the aforementioned public suffix list; there are quite a few entries along these lines there already, including operaunite.com or appspot.com - but this approach obviously does not scale particularly well. The list is also not supported by all existing browsers, and not mandated in any way for new implementations.

"Oh, please. Nobody is actually going to depend on them."

In the RFC 2109 paragraph cited earlier, the specification pragmatically acknowledged that implementations will be forced to limit cookie jar sizes - and then, confusingly demanded that no fixed limits are put in place, yet specified minimum limits that should be obeyed by implementators.

What proved to be missing is any advice on a robust jar pruning algorithm, or even a brief discussion of the security considerations associated with this process; any implementation that enforces the recommended minimums - 300 cookies globally, 20 cookies per unique host name - is clearly vulnerable to a trivial denial-of-service attack: the attacker may use wildcard DNS entries (a.example.com, b.example.com, ...), or even just a couple of throw-away domains, to exhaust the global limit, and have all sensitive cookies purged - kicking the user out of any web applications he is currently logged into. Whoops.

It is worth noting that given proper warning, browser vendors would not find it significantly more complicated to structure the limits differently, enforce them on functional domain level, or implement pruning strategies other than FIFO (e.g., taking cookie use counters into account). Convincing them to make these changes now is more difficult.

While the ability to trash your cookie jar is perhaps not a big deal - or rather, the ability for sites to behave disruptively is also poorly mitigated on HTML or JavaScript level, making this a boring topic - the weakness has special consequences in certain contexts; see next section for more.

Be my very special cookie

Two special types of HTTP cookies are supported by all contemporary web browsers: secure, sent only on HTTPS navigation (protecting the cookie from being leaked to or interfered by rogue proxies); and httponly, exposed only to HTTP servers, but not visible to JavaScript (protecting the cookie against cross-site scripting flaws).

Although these ideas appear to be straightforward, the way they were specified implicitly allowed a number of unintended possibilities - all of which, predictably, plagued web browsers through the years. Consider the following questions:

• Should JavaScript be able to set httponly cookies via document.cookie?

  
  

• Should non-encrypted pages be able to set secure cookies?

  
  

• Should browsers hide jar-stored httponly cookies from APIs offered to plugins such as Flash or Java?

  
  

• Should browsers hide httponly Set-Cookie headers in server responses shared with XMLHttpRequest, Flash, or Java?

  
  

• Should it be possible to drop httponly or secure cookies by overflowing the "plain" cookie jar in the same domain, then replace them with vanilla lookalikes?

  
  

• Should it be possible to drop httponly or secure cookies by setting tons of httponly or secure in other domains?
  

All of this is formally permitted - and some of the aforementioned problems are prevalent to this day, and likely will not be fixed any time soon.

At first sight, the list may appear inconsequential - but these weaknesses have profound consequences for web application design in certain environments. One striking example is rolling out HTTPS-only services that are intended to withstand rogue, active attackers on open wireless networks: if secure cookies can be injected on easy-to-intercept HTTP pages, it suddenly gets a whole lot harder.

If it tastes good, who cares where it comes from?

Cookies diverge from JavaScript same-origin model in two fairly important and inexplicable ways:

• domain= scoping is significantly more relaxed than SOP, paying no attention to protocol, port number, or exact host name. This undermines the SOP-derived security model in many compartmentalized applications that also use cookie authentication. The approach also makes it unclear how to handle document.cookie access from non-HTTP URLs - historically leading to quite a few fascinating browser bugs (set location.host while on a data: page and profit!).
  

  
  

• path= scoping is considerably stricter than what's offered by SOP - and therefore, it is completely useless from the security standpoint. Web developers misled by this mechanism often mistakenly rely on it for security compartmentalization; heck, even reputable security consultants get it completely wrong.
  

On top of this somewhat odd scoping scheme, conflict resolution is essentially ignored in the specification; every cookie is identified by a name-domain-path tuple, allowing identically named but differently scoped cookies to coexist and apply to the same request - but the standard fails to provide servers with any metadata to assist in resolving such conflicts, and does not even mandate any particular ordering of such cookies.

This omission adds another interesting twist to the httponly and secure cookie cases; consider these two cookies:

Set on https://www.example.com/:
  FOO=legitimate_value; secure; domain=www.example.com; path=/

Set on http://www.example.com/:
  FOO=injected_over_http; domain=.example.com; path=/

The two cookies are considered distinct, so any browser-level mechanisms that limits attacker's ability to clobber secure cookies will not kick in. Instead, the server will at best receive both FOO values in a single Cookie header, their ordering dependent on the browser and essentially unpredictable (and at worst, the cookies will get clobbered - a problem in Internet Explorer). What next?

Character set murder mystery

HTTP/1.0 RFC technically allowed high-bit characters in HTTP headers without further qualification; HTTP/1.1 RFC later disallowed them. Neither of these documents provided any guidance on how such characters should be handled when encountered, though: rejected, transcoded to 7-bit, treated as ISO-8859-1, as UTF-8, or perhaps treated in some other way.

The specification for cookies further aggravated this problem, cryptically stating:

"If there is a need to place such data in the name or value, some encoding method such as URL style %XX encoding is recommended, though no encoding is defined or required."

There is an obvious problem with saying that you can use certain characters, but that their meaning is undefined; the systemic neglect of this topic has profound consequences in two common cases where user-controlled values frequently appear in HTTP headers: Content-Disposition is one (eventually "solved" with browser-specific escaping schemes); another is, of course, the Cookie header.

As can be expected, based on such poor advice, implementators ended up with the least sensible approach; for example, I have a two-year-old bug with Mozilla (418394): the problem is that Firefox has a tendency to mangle high-bit values in HTTP cookies, permitting cookie separators (";") to suddenly materialize in place of UTF-8 in the middle of an otherwise sanitized cookie value; this led to more than one web application vulnerability to date.

A session is forever

The last problem I want to mention in this post is far less pressing - but is also an interesting testament to the shortcomings of the original design.

For some reason, presumably due to privacy concerns, the specification decided to distinguish between session cookies, meant to be non-persistent; and cookies with a specified expiration date, which may persist across browser sessions, are stored on the disk, and may be subject to additional client-enforced restrictions. On the topic of the longevity of the former class of cookies, the RFC conveniently says:

"Each session is relatively short-lived."

Today, however, this is obviously not true, and the distinction feels misguided: with the emergence on portable computers with suspend functionality, and the increased shift toward web-oriented computing, users tend to keep browsers open for weeks or months at a time; session cookies may also be stored and then recovered across auto-updates or software crashes, allowing them to live almost indefinitely.

When session cookies routinely persist longer than many definite-expiry ones, and yet are used as a more secure and less privacy-invasive alternative, we obviously have a problem. We probably need to rethink the concept - and either ditch them altogether, or impose reasonable no-use time limits at which such cookies are evicted from the cookie jar.

Closing words

I find it quite captivating to see the number of subtle problems caused by such a simple and a seemingly harmless scheme. It is also depressing how poorly documented and fragile the design remains some 15 years later; and that the introduction of well-intentioned security mechanisms, such as httponly, only contributed to the misery. An IETF effort to document and clarify some of the security-critical aspects of the mechanism is underway only now - but it won't be able to fix them all.

Some of the telltale design patterns - rapid deployment of poorly specified features, or leaving essential security considerations as "out of scope" - are still prevalent today in the browser world, and can be seen in the ongoing HTML5 work. Hopefully, that's where the similarities will end.

Read More