The old quagga AS4 patch

• 19.10.2007 Two days ago, I received the news that quagga AS4 is suspectible to two kinds of DOS attacks if a wrong set of attributes or a wrongly formatted AS4_PATH is received. Both things are only possible by a faulty bgp/rfc4893 implementation or real malicious intend (a software that injects something like that on purpose). Although I already stated the end of the AS4 patch four days ago, I think you are right to expect a reaction in this special case. So, you can read the original mail about this and download a fix for the holes quagga-AS4 has open there.

• 15.10.2007 As of today, quagga mainline CVS has AS4 support built-in. So please note that quagga cvs 20071014 is the latest quagga you can use this patch on! The work on AS4 I have done is finished by this milestone.

  I will provide one (ONE!) patch in order to have complete freedom of asdot representation in quagga again - something which Paul dumped because he is opposed to it and thinks asplain is the better choice. He is not alone in that regard. But there is the other party encompassing the RIRs and the pioneer AS4 users, who are just as keen on using asdot as Paul et all are on asplain. So what!? There are reasons for both, though I personally have never seen a reason to finally choose either one (from the implemention point of view).

  I do not have the time or enthusiasm to support a CLI patch for a number representation: it is tedious, and nothing fancy is achieved by it. By providing one patch for the already used syntax I hope to give a final service to "my" users. CLI changes are not very common (excepting additions ;-)) so a patch should be good for quite a while. Do not hold your breath for this one, I see no immediate need as long as no *big* issues are added into quagga cvs after the AS4 patch, so I will take some time for doing this.

  Having said that, I think I have finally come to the conclusion what would have been the right choice for adding AS4 support, but it is too late now to go that path: Choosing *any* representation without user interaction is wrong when there are several representations to choose from. The right way IMHO would have been to add the whole AS4 stuff, and show only AS_TRANS to the user in place of any 4 byte as number, and give her no way to enter or see any AS4 number in the CLI at all. The whole AS4 stuff could thus have been hidden totally from the user, who stays in the AS2 world although quagga would have been already truly AS4 'under the hood'. The true 4-byte as numbers a user can enter and see if and only if she gives the command which chooses whether asplain, asdot, or asdot+ is to be used - which will of course then be saved in the configuration. That way, the user has made the choice, no default forces her (or even pushes her slightly) to use something and any incompatibilites regarding regular expressions are completely in the users domain. That is what I think is good, open software: The user decides what is used, and any choice which is deemed sensible by more than a handful of people is there. But: too late. The best ideas you always get after you have no way to use them any more...

• Patch v09 released on 13.09.2007
• 10.09.2007 Quagga 0.99.9 has come out, and there have been some changes since 0.99.7. 0.99.9 has not got AS4 support yet - so I will be providing a version v09 for the quagga CVS version 0.99.9-20070909 (i.e. the version with the many 9s), skipping v08 for purely aesthetic reasons. Because of the changes in quagga I'll have to do some basic tests again before releasing v09, but you can expect it in the upcoming days. And yes, I am still living and maintaining this ;-) though other things kept me busy for a while.
  Regressions in quagga 0.99.9
  Be aware that quagga 0.99.9 and cvs before 20070918 have two regressions - you can not announce ipv6 routes, and if you get an ipv6 default route quagga will take down the session it is received on. Workaround for the second is - do not let your peer send you the default. Bug fix for the first is in cvs20070918, which should have no problems with the v09 patch.
• I am sorry that this site was down for a while. Someone mailed me that it was down... otherwise I'd have noticed that even later :-/.
• Patch v07 released on 17.05.2007
• existence of patch was posted on quagga-users and nanog (by others)
• patch is listed in quagga-resources
• most search engines find this with the terms "AS4 quagga".
• positive feedback received (see further down).
• waiting for further feedback and/or experiences from the community.
• basic AS4 handling stable through the first 6 versions, changes were in naming and in the "add-ons" (extended communities and MRT dumps). First enhancements in basic AS4 handling present in v07, read further down.
• One BUG present in all 07 versions, bug fix available, read further down.

Newsflash 28.03.2007:
All of RIPE NCCs RIS RRCs have been moved to Quagga 0.99.6 with AS4 patch v05 today!
Which means that all RIPE NCC Looking Glasses now show you true AS4 numbers (i.e. try the regexp "\." <backslash><dot>).
And it is not only the Looking Glasses, it is the whole RIS project, read Erik's feedback.

Patch release

Documentation

1. AS numbers which are larger than 65535.
   So you need to know about the format as numbers are in and the related quagga configuration command.
2. MRT dumps which contain 4 byte AS numbers.
   You need to be aware that quagga uses a new format for this, and read draft-ietf-grow-mrt-04.txt if you use MRT dumps.
3. Extended communities with 4 byte AS numbers.
   The format on the wire for this is defined in draft-rekhter-as4octet-ext-community-01.txt , and the CLI format you'll have to use to instruct quagga to use this (and what to specify if you only use the old 2-byte-AS extended communities) is described by extended communities and quagga AS4 .
4. Debugging AS4.
   Sometimes, you'll want to know what happens, either just to see what's going on or to do some debugging, and quaggas new as4 specific debugging commands and their output may move you along in that case.
5. Basic AS4 handling.
   If you really want to read the draft which defines the basic AS4 handling, you'll have to dig into RFC4893 (which was draft-ietf-idr-as4bytes-13.txt in former times) - but that is not necessary to use this patch.
6. Other AS4 background.
   And if you have to much time and wonder why you have a choice of several as number formats now, and why you as the user have to specifiy whether to use an extended community for a 2 byte as or the one for a 4 byte as, then you can read my thoughts on as4.
7. History of this patch.
   You can read about this in the log. Things which happened in 2006 are now in the 2006 log.

Notes

Patch base

The newest patch version 09 [available ASAP] (there is NO version 08) is based on cvs quagga 20070909, which is quite identical to "official" 0.99.9. The only differences between v09 and v07 are: the ibgp bug is fixed, and the patch base has changed. If you are using 0.99.7 or older versions of quagga and want to patch them, you are better off sticking to version v07 (with the ibgp-patch applied).

The patch 07 may even apply to earlier versions, but you are on your own there. Applying 09 to earlier versions will run into problems. Because of 09 being identical (AS4 semantic-wise) to 07 plus ibgp-patch, there is no incremental patch from 07 to 09.

Be aware that there have been some changes in the internal quagga structures between 20070430 (which v07 was based on) and 20070909, and v09 is basically untested for now, so be careful, test it, and please send feedback if it runs as smoothly as the other versions did. Of course, v09 survived my small testbed ;-).

Contents of the patch

The as4 patch changes *.[ch] files in the bgpd/ subdirectory of quagga and a few bits in the doc/, tests/, and vtysh/ subdirectory as everybody should have expected. No new files were added, no file removed. Thus, no problems with the build process of quagga - whichever one you may use (debian package, redhat package, build with configure from source without any packaging) - are to be expected, I hope.

Semantics implemented

All actions implemented are defined in RFC4893 (which was draft-ietf-idr-as4bytes-13.txt in former times). If anything shows up which is not handled as it is described in the RFC, it is my fault: email me, and I'll try to correct the issue.
Quagga can use extended communities with 4 byte as numbers in them, using draft-rekhter-as4octet-ext-community-01.txt for the bits "on-the-wire". For the syntax please refer to this documention; you have to explicitly tell quagga to use the 4 byte wire-format. Be aware that earlier versions used another syntax which encouraged misinterpretations which is why I junked that. The current available drafts for extended communities leave something open, if you want to have some background on this, feel free to read my thoughts on as4, but you will need a bit of time for them.

Caveats

The general one: I may have goofed in a lot of places. If you try this patch, you do this on your own risk. This patch may crash your quagga and all routers you peer with, but that is entirely *your* risk. I have tried to give you something which does not do these things, but YMMV.
A specific one: All things regarding AS_CONFED_* are pretty much untested. If you run a confederation you are more prone to stumble upon something than you are if you are not running one.

New commands

There is a new configuration command 'bgp asnum format (asdot|asdot+|asip|asplain)' which sets the output format for as numbers. The different formats are documented here. The format of as numbers has implications for your router configurations. I elaborated a bit on that in my thoughts on as4, but you will need a bit of time for reading them .
There are two new debug commands 'debug bgp as4' and 'debug bgp as4 segment' which will try to fill up your logfile with hopefully useful information if you enable them. The 'segment' debug is unsuited to a full bgp routing table as it logs several lines for each route received, so take care there. You will find all the messages which are logged if you use these debug commands as well as errors logged by the AS4 handling documented here.

Bugs?

If you find such, tell me, I'll try to find a solution. But feed only as4 related bugs to me, please. And do test this thing before going productive with it, and also tell me if you have no problems at all (as soon as this patch is in quagga mainline, no further positive feedbacks will be necessary).

Feedback

Send any feedback by email. Success stories would be very welcome ;-). If you do not mind it specifically when giving feedback, I'll put relevant feedback right here on this web page.

RIPE-54

I gave a presentation regarding quagga and the AS4 patch at RIPE-54 in Tallinn on May 11th, 2007. The presentation is available via the RIPE website, but you can also directly download it (296674 Bytes). The webcast of the presentation can also be downloaded from the RIPE website.

DE-CIX technical meeting 20th September, 2007

I gave another presentation regarding RFC 4893 and the quagga AS4 patch at the DE-CIX technical meeting on 20th September 2007. That presentation is more-or-less the same as the Tallinn one, a bit beefed up and updated - and you may download the slides as pdf here.

Feedback and/or bugs/enhancements in released version(s) of the patch itself

11.09.2007

I am busy putting together v09, based on cvs20070909, which will also be usable for official 0.99.9. In order to have more "9"s, I have skipped number 08: there is and will be no version v08 ;-).

24.07.2007

Coming back from my summer holidays on the Outer Hebrides I met Paul Jakma in Glasgow and we had a nice afternoon talking about quagga. The right way to go regarding as number representation in quagga with AS4 is still debated between us, but we know (and have always known - just so that nobody gets other ideas) that we both want to have the "best" solution for the users. "Best" is very, very shady if you look upon the representation. There may be no best solution at all... That said, Paul had also already started the integration effort into mainline - so that is in the works, too - but do not hold your breath.

04.07.2007

Incoming bug report. There is a setup in which the as-path synthesizing (or do you call it reconciliation) dumps core. Found and reported by Katsuyasu Toyama, who also supplied a correct patch for correcting the problem which was found by his collegue Arifumi Matsumoto:
Bug (all patch version affected): if an AS4 quagga is talking with iBGP inside an AS to AS2 speakers (the AS the iBGP is taking place being an AS2) and one of the AS2 speakers has an AS4 as an external neighbor, quagga AS4 dumps core.
Reason: Access to uninitialized memory (yuck!). The corresponding execution path excaped me during coding and testing - and it took a while to hit someone, too.
Solution: Apply this patch (1683 Bytes) to your AS4 quagga sources! Should patch cleanly into version 07, and you may ignore any conflict in the Changes files for other versions.
Workaround: Let an AS4 capable quagga speak to the external AS4 directly and let all AS4 capable quaggas talk to each other (i.e. not "over" a non-AS4 route-reflector).

25.05.2007

Received positive feedback from Gert Doering, who is currently involved in setting up his AS3.3 on FreeBSD.

17.05.2007

Version v07 released. This release makes the first real changes in the core AS4 routines, and is thus based on the same CVS release as version v06 to have things plainly in sight.

There are two changes: burn less memory, and send attributes in ascending order.

Burn less memory
Versions 01-06 had extended the attribute structure by the AS4 values as4_path, as4_aggragator and as4_aggregator_address - which increased the structure by 12 bytes. Because the AS4 path was always freed after all the necessary work was done the AS4 path never needed more memory than that.
But the AS4 path handling does not need the remaining 12 bytes either, so I chucked them out of the structure again. This is in step with Paul's effords to save memory where possible, and will make it easier to re-base the AS4 patch after Pauls attribute structure changes because the AS4 patch now needs no changes in the structure at all. And, if you have 500000 routes with attributes in memory (say, full DFZ from two places), this will save 6 meg memory!

Send attributes in ascending order
Versions 01-06 send the AS4_PATH directly after AS_PATH, and AS4_AGGREGATOR directly after AS_AGGREGATOR. This was done because the fact that AS4 stuff was to be send is determined when the AS2 counterparts are dealt with very easily.
RFC4271 states that "the sender of an UPDATE message SHOULD order path attributes within the UPDATE message in ascending order of attribute type", something which quagga did not do in doing it as it did. Of course, that is only a "SHOULD", not a must, so the implementation had the freedom to do it as it did (no violation of the RFC, but using possible leeway).
Because it is not a big change to send the attributes in ascending order of attribute type I changed the code to do just that.
This may help people out there who have routers which have a problem if attributes do not arrive sorted the right way.

14.05.2007

Kevin Loch and SUZUKI Shinsuke have informed me that the AS4 patch is acting against this "SHOULD" in the first sentence of this part of RFC4271:
"The sender of an UPDATE message SHOULD order path attributes within the UPDATE message in ascending order of attribute type. The receiver of an UPDATE message MUST be prepared to handle path attributes within UPDATE messages that are out of order."
Interestingly, this has pointed out some routers violating the MUST in the second sentence.

I'll handle this very useful feedback as a feature request: the next version of the quagga AS4 patch will send AS4_PATH and AS4_AGGREGATOR at the correct place. I'll also provide for an 'update patch' to correct older versions of already AS4 patched quagga sources in regard to that.
The current sending position of these attributes has been motivated by implementation simplicity only, but there is no deeper reason why quagga should continue to ignore the "SHOULD". The changes are pretty straightforward, so I expect no difficulities in resolving the issue. Expect version v07 pretty soon.

But this means: if you start using the AS4 patch in a version up to and including v06 and one of your peers becomes unstable at that time, you may have the same problem: a peer (router) which relies on the order of attributes and can not cope with attributes coming in not quite sorted.

04.05.2007

Version v06 released. Mainly a re-base onto quagga cvs 20070430, which mostly resembles quagga-0.99.7; on top of the re-base, the "statsbugfix" (see below) is included and some left-overs of the term "...as4_aspath..." have been changed to "...as4_path...". So, the reason for releasing this as a new version is a more or less to provide a "clean" patch when you start with 0.99.7. There were a some changes in the cvs version of bgpd in the last weeks resulting in a lot of offsets if you use v05 on it, and I try to make your life as easy as possible.

There is an incremental patch from v05 to v06 here, too: remember that the incremental patches DO NOT upgrade you to a current cvs version or to the 0.99.7 version. The incremental patches encompass changes in the AS4 stuff ONLY.

12.04.2007

"32-bit AS numbers in the wild" is an article on http://www.bgpexpert.com which links back to this site. Note that Geoffs route (203.10.62.0/24 from 2.2) vanished only for a few days - probably collecting Easter eggs ;-). Btw, the re-appearing has been noticed by Jac Kloots who also provides a mrtg statistics on AS4 originated ipv4 prefixes present in the DFZ.

10.04.2007

Jac Kloots tells nanog that SURFnet is using AS4 quagga and is doing probably the first ipv6 announcement originating from a 4 byte AS.

02.04.2007

Cosmetical bug in all versions (found by Jac Kloots, who will also start announcing an ipv6 prefix from 3.5 soon):
'sh ip bgp unicast statistics' always shows the Highest public ASN in asplain.
Reason: That there is an ASN in the statistics completely eluded me - in fact it is stored in something called "counts", so no way to grep for this ASN output... My fault, no excuses ;-).
Solution: available, use v06 or later.
Or use this interim short patch to update any earlier version regading this.

29.03.2007

A feedback update from Jac Kloots arrived: He changed filters, and the SURFnet route 145.125.0.0/20 is now visible worldwide as coming from 3.5. He also says that "it seems all works fine until now."
There are now 3 routes visible worldwide originating from 4 byte as numbers (listed here in the order they appeared):

• Geoffs beacon (203.10.62.0/24 from 2.2),
• SURFnets /20 (145.125.0.0/20 from 3.5), and
• RIPE NCCs beacon (84.205.88.0/24 from 3.7).

Hm, Geoff said he switched to quagga so it is quite possible that all three prefixes are quagga-originated!

28.03.2007

Erik Romijn tells me he has switched all of RIPE NCCs RIS boxes to quagga 0.99.6 with AS4 patch v05, and "everything seems to be working fine".
If this runs without problems for some weeks we can all be sure that the AS4 patch is solid as a rock - I hope so, of cause ;-).

22.03.2007

Jac Kloots made an explicit feedback out of the implicit one I mentioned on 09.03.2007. Many thanks for the feedback!

14.03.2007

Version v05 released. Fixes the uncleanliness in extended community handling, the hackiness of the mrt dumps, and some other unclean thingies.
Solves: semantic impact of as number input format for extended communities, non-draft (aka non-standard) MRT dump format.

09.03.2007

Implicit feedback: Yesterday I saw one of the routes Jac Kloots was talking about last month in the RIS looking glasses as coming from AS_TRANS (second route in the world doing that). So I'd presume SURFnets "3.5" has entered the real world using quagga.

13.02.2007

Erik Romijn and Geoff Huston are busy testing AS4 in relation to RIS. Geoff did some inter-operation AS4 testing with quagga and openBGPD, and provided this feedback: openBGPD has problems coping when presented with a full routing table, but quagga has not.

12.02.2007

Jac Kloots (SURFnet) tells me that SURFnet has got AS 3.5 from RIPE NCC, and he is setting up an AS4 patched quagga which will start advertising a route soon.

09.02.2007 (found by Erik Romijn, RIPE NCC):

Bug in versions v01-v03 of this patch: vtysh can not cope with 'router bgp ASNUMBER' for all asnumbers possible.
Reason: Left vtysh completely alone when coding the patch. And that command must be dealt with in vtysh, too.
Solution: available, patch version 4, posted 2007-02-12
Workaround for versions 1-3: Do not use vtysh for bgp configuration, use telnet to bgpd directly.

17.01.2007

Erik Romijn, RIPE NCC, provided a first feedback for the patch on quagga-users.

16.01.2007 (found by myself, after a related question from Erik Romijn):

Bug in v01 and v02 of this patch: The commands 'ip extcommunity-list <1-99>|standard' do not have ASN in their syntax, they have the old 'aa' there. Further bug: Printing out an extcommunity list gets rid of "0." if you define an asn32 extcommunity with an asn16 in it, so asn32 extcommunities can not be differentiated from asn16 ones. Saved configurations will then loose the information that an asn32 extcommunity has been supplied. (But, who will use an asn32 extcommunity when the as number in question is smaller than 65536?) Another solution would be to enforce an asn16 ecommunity when an asn16 is given with leading "0." on input (would be logical). Guess a completely different syntax (third solution) or even doing some more groundwork for extcommunities would be best - something to discuss! You will find some more explanation about what I am ranting about here in my (rather longish) thoughts on as4.
Reason: Forgot to code that :-(
Solution: available (force asdot+ output for asn32 extcommunities), patch version 3, posted 20070118
Better solution: available (change syntax for extended commmunites), patch version 5, posted 20070314

11.01.2007 (found by Erik Romijn, RIPE NCC):

Bug in v01 of this patch: MRT dumps do not work, wrong data in as numbers.
(One) Reason: bgp_dump.c still handels as numbers as only 16 bit.
(Another) Reason: bgp_dump.c does not set the right message types for the 32bit things. Actually, some needed values are not officially defined yet... At least, that is an excuse for me not providing something working in the first place for this ;-)
Solution: available, patch version 2, posted 20070115.
Be aware that the MRT patches are still a bit preliminary. 32bittiness is hacked into the dumps in the afi-fields, which is very ugly. A discussion has been started on the GROW wg (the mailing list is here), and as soon as there are results, you will probably find another patch version here. From the operational point of view this is sugar... but with this hacked version we will probably have only libbgpdump which is able to read this format. Well, as far as we know there is noone else who implemented asn32 dump reading anyway.
And a note for the folks knowing nothing whatsoever about MRT dumps: this whole MRT dump stuff has no relationship with the basic asn32 semantics, it is only for dumping bgp information(s) to file.
Newer solution: available (follow the new version of the corresponding draft), patch version 5, posted 20070314.

Download area

Remember that the patch base is the stated cvs snapshot version of quagga, not "the 0.99.6 version" or "the 0.99.7 version". Having said that, official 0.99.6 resembles cvs version 20061208 and official 0.99.7 *mostly* resembles version 20070430. As long as no big changes appear in the bgpd subdirectory of cvs quagga, you will probably get away with minor things like conflicts in Changelogs or such if you use the newest patch on plain 0.99.7.

To apply a patch, change into the directory of a quagga version, and call 'patch -p0 < PATCHFILE', where PATCHFILE is the patch you downloaded.

Version 05 of the patch changes the syntax for extended community values, so please read this.
Furthermore, version 05 is now able to do MRT dumps as defined in draft-ietf-grow-mrt-04.txt , which came out on 20070307, so we are very up-to-date regarding this.

Always use the newest version, the older versions are only here for reference.

Whichever version you use (hopefully v07), you must also download this bug fix patch (1683 Bytes) in order to get no known bad surprises inside your running AS2 (see the bug list above).

The incremental patches are for the people who have already started to work with this. If you would rather patch in the missing things than starting again with a CVS quagga, you can upgrade your version with these patches. Be aware that you miss changes between the cvs versions of quagga when the cvs patch base changes between two patch versions (i.e. from 04 to 05). This may or may not be what you want.

Incremental Patch	Remarks
quagga-asn32-v01-v02-inc.patch (5090 Bytes)	This brings you from version v01 to version v02.
quagga-asn32-v02-v03-inc.patch (3700 Bytes)	This brings you from version v02 to version v03.
quagga-as4-v03-v04-inc.patch (48550 Bytes)	This brings you from version v03 to version v04. 1 offset of -8 lines in bgpd/vtysh.c may be ignored.
quagga-as4-v04-v05-inc.patch (32526 Bytes)	This brings you from version v04 to version v05.
quagga-as4-v05-v06-inc.patch (9057 Bytes)	This brings you from version v05 to version v06.
quagga-as4-v06-v07-inc.patch (11116 Bytes)	This brings you from version v06 to version v07.

There is no incremental patch from v07 to v09, because there were changes in quagga from 0.99.7 to 0.99.9 which changed structures, so you can not use the changes from v07 to v09 without going from 0.99.7 to 0.99.9 anyway.

Encountered bugs in quagga cvs

Todo

Provide more documentation:

Say something about the MRT patches, i.e. put the relevant parts of GROW somewhere, so people know the format they are in.

Change patch to a format suitable for inclusion into mainstream quagga:

Clean out comments big way, do not clutter the code e.g. with parts of the draft or loud thoughts. This will probably cause the patch to shrink to half its size.
Brush up some of the #defines - some usages are unreadable as-is.
Try to break this whole thing into pieces, extcommunities and the MRT part can be done in their own patches (they have no influence to the basic AS4 awareness).
Go to one representation of as numbers anywhere, leave no choice. Can be done as soon (as late!) as the whole world agrees on what to use. (This basically puts inclusion into mainstream quagga quite far in the future.)

Log

-- Juergen
       Feedback: j.kammer (at) eurodata.de