Sample quotes from the hardworking anonymous reviewers

• Reviewer 1: "The paper considers a real problem and proposes an interesting and much more efficient solution than the current state-of-the-art. The presentation of the proposed solution is thorough and detailed, as well as its performance evaluation."
• Reviewer 1: "The paper is interesting because it considers a real problem (access control policy through XACML) and proposes a nice solution that uses a series of simplifying transformations. I personally enjoyed the style of Section 3, posing the problem and then the solution, as they appear."
• Reviewer 2: "The paper introduces an engine that is much more efficient at processing access control policies. The approach could have a significant impact, should it be adopted, as we are likely to see more rather than fewer access control policies in the future, particularly as more services are deployed and utilized on the Internet."
• Reviewer 2: "The performance of the engine is significantly faster (3-5 orders of magnitude) than an existing engine. The engine could become widely deployed."
• Reviewer 2: "I liked the paper in that it provides the intuition behind the significant performance improvements. The performance improvements are also impressive."
• Reviewer 3: "Overall, this was my favorite paper among the 15 I have reviewed."
• Reviewer 3: "This paper describes a highly tuned engine for evaluating XACML policy queries. Results show that the engine is one to five orders of magnitude faster than the state-of-the-art Sun PDP engine."
• Reviewer 3: "The performance of the system is very impressive. It was not a priori obvious such normalization would be practically possible. The algorithmic descriptions in the paper are highly detailed. The evaluation is very carefully constructed, and it includes a correctness validation as well as true apples-to-apples performance comparison."
• Reviewer 3: "I am not normally a fan of "we have improved the performance of the Foobar system by X" papers, but I'll certainly make an exception when the value of X is several orders of magnitude. I am even more ready to make an exception when the paper is the first serious study of performance improvement in a widely applicable system with novel characteristics relative to previously studied systems. This paper is exactly that, and I wholeheartedly endorse its acceptance."
• Reviewer 3: "The paper is highly detailed. I wish I could have afforded to spend more time digging into its details. For most papers I've reviewed, three or four hours has been plenty of time to digest the details of the work, but for this paper that was not sufficient time, at least not for me."
• Reviewer 3: "The XACML language is insanely complicated. As the authors note, this degree of complexity "...makes brute force searching appear to be the natural way of processing requests." It really was not apparent to me that the degree of normalization the authors have achieved was practically possible. Although some of their techniques are obvious, such as the numericalization of string tokens, others are surprisingly involved and subtle, such as the procedure for converting range rules to permit-deny ordering."
• Reviewer 3: "A note on my ratings: Overall, the papers I reviewed for Sigmetrics this year have been of strikingly high quality. In holding myself to the ratings curve (top 10%, top 25% but not top 10%, etc.), I have found that I am giving noticeably lower ratings than in previous years for the same quality of work. This increased competitiveness is generally a good thing for Sigmetrics and for the community, but it can be rather hard on the authors of individual papers. This is particularly true for my Readability ratings; since most papers were quite well written, the rating you see above is not at all damning."