Amazon Product Api REST Client in Scala

I have finally decided to take a stab at Scala. I have been reading about it for a while now, and coming from a Java Environment it's very appealing to me.

What better start than the Amazon Product API authentication example? Since I recently struggled with it. It seems easy and straightforward.

I will display only the equivalent of the SignedRequestsHelper class, since that is doing all the hard work.

import _root_.java.net.URLEncoder;
import _root_.java.text.DateFormat;
import _root_.java.text.SimpleDateFormat;
import _root_.java.util.Calendar;
import _root_.java.util.TimeZone;

import _root_.javax.crypto.spec.SecretKeySpec;
import _root_.javax.crypto.Mac;

import _root_.org.apache.commons.codec.binary.Base64;

import scala.collection.immutable.SortedMap;
import scala.collection.immutable.TreeMap;

trait SignedRequestsAmazonApi {
  
      val awsAccessKeyId = "YOUR ACCESS KEY";
    val awsAssociateTagKey = "YOUR TAG";
    val awsSecretKey = "YOUR SECRET KEY";
 
      val UTF8_CHARSET = "UTF-8";
    val HMAC_SHA256_ALGORITHM = "HmacSHA256";
    val REQUEST_URI = "/onca/xml";
    val REQUEST_METHOD = "GET";

    val endpoint = "ecs.amazonaws.com";
    val service = "AWSECommerceService";
    val version = "2009-07-01";
 
    val secretKeySpec = new SecretKeySpec(awsSecretKey.getBytes(UTF8_CHARSET), HMAC_SHA256_ALGORITHM);
 
    def sign(params: Map[String, String]) :String = {
      
        var sortedParamMap = TreeMap.empty[String, String]
        sortedParamMap = sortedParamMap.insert("AWSAccessKeyId", awsAccessKeyId)
        sortedParamMap = sortedParamMap.insert("AssociateTag", awsAssociateTagKey)  
        sortedParamMap = sortedParamMap.insert("Service", service)
        sortedParamMap = sortedParamMap.insert("Version", version)
        sortedParamMap = sortedParamMap.insert("Timestamp", timestamp())  
        params foreach ( (o) => sortedParamMap = sortedParamMap.insert(o._1, o._2))
  
        val canonicalQS = canonicalize(sortedParamMap);
        val toSign = REQUEST_METHOD + "\n" + endpoint + "\n" + REQUEST_URI + "\n" + canonicalQS;
        val hmacStr = hmac(toSign);
        val sig = percentEncodeRfc3986(hmacStr);

        "http://" + endpoint + REQUEST_URI + "?" + canonicalQS + "&Signature=" + sig;
    } 
 
     protected def hmac(stringToSign: String) :String = {
        
        try {

            val mac = Mac.getInstance(HMAC_SHA256_ALGORITHM);
            mac.init(secretKeySpec);

            val data = stringToSign.getBytes(UTF8_CHARSET);
            val rawHmac = mac.doFinal(data);
   
             val encoder = new Base64();
             return encoder.encodeToString(rawHmac).trim();

        } catch {
              case _ => throw new RuntimeException(UTF8_CHARSET + " is unsupported!")
        }
    }
 
    protected def timestamp(): String = {
      
        val dfm = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
        dfm.setTimeZone(TimeZone.getTimeZone("GMT"));
        return dfm.format(Calendar.getInstance().getTime());
  
    }
 
    protected def canonicalize( sortedParamMap: SortedMap[String, String]): String = {
      
      sortedParamMap.map[String] { 
        (key) => percentEncodeRfc3986( key._1 ) + "=" + percentEncodeRfc3986(key._2) 
      }.reduceLeft[String]{ (acc, url) => acc + "&" + url }
      
    }
 
    protected def percentEncodeRfc3986(s: String): String = {
        try {
            return URLEncoder.encode(s, UTF8_CHARSET).replace("+", "%20").replace("*", "%2A").replace("%7E", "~");
        } catch  {
            case _ => throw new RuntimeException(UTF8_CHARSET + " is unsupported!")
        }
    }
}

This will return the encoded / encrypted url that you need to GET
And to test all that:

import scala.collection.immutable.TreeMap;

class TestApi extends SignedRequestsAmazonApi{
  
  def test(){
    var params = TreeMap.empty[String, String]
    params = params.insert("Operation", "ItemSearch")
    params = params.insert("SearchIndex", "Books")
    params = params.insert("Keywords", "harry potter")
  
    val requestUrl = sign(params);
    Console.out.println("Signed Request is \"" + requestUrl + "\"");
  
  }

As I've said, it seems very easy to do in Scala.
I chose to implement that as a Scala Trait, and I'm sure there are many things that can be done better here.

Authenticating to Amazon Product API in Java. The notorious %0D%0A

Today I stumbled upon a small bug in the Amazon Authentication Example for Java sample (you can replace 'stumbled upon' with 'banged my head on the wall for some hours' and you'll get an idea about what I went through ;)

It seems the example provided by Amazon does not work. Try it and you get the following error: Server returned HTTP response code: 403 for URL.

You can even look here at the Signed Requests Helper but it will not help you with the Java problem in the code. Especially because you must encode the entire url which has a timestamp in it, so it will change every time, making it close to impossible to compare.

After digging around I managed to locate the problem.It's in the SignedRequestsHelper.java class, the 'private String hmac(String stringToSign)' method.
The issue seems to be that the signature String returned contains an extra %0D%0A chars. It seems that the Base64 encoder just adds that to the return value. The quick fix here is to just add a simple trim() to the return string, in order to remove the trailing chars.

The new and improved method looks like:

  /**
   * Compute the HMAC.
   * 
   * @param stringToSign
   *            String to compute the HMAC over.
   * @return base64-encoded hmac value.
   */
  private String hmac(String stringToSign) {
    String signature = null;
    byte[] data;
    byte[] rawHmac;
    try {
      data = stringToSign.getBytes(UTF8_CHARSET);
      rawHmac = mac.doFinal(data);
      Base64 encoder = new Base64();
      
      // notice the use of the improved method encodeToString() from the
      // Base64 class AND our very important fix 'trim()'
      signature = encoder.encodeToString(rawHmac).trim();  

    } catch (UnsupportedEncodingException e) {
      throw new RuntimeException(UTF8_CHARSET + " is unsupported!", e);
    }
    return signature;
  }

Full disclosure, I am using the newest version of the new Apache Commons Codec lib - the 1.4 one. I am unsure if this is a issue, it really should not be.

Hopefully this will save someone a few hours of digging through the Amazon samples. I just started using their services and it seems like a lot of information to digest.