SSLPeerUnverifiedException and debugging SSL connections

I’ve faced the problem of having to deal with a strange exception seen in our application log:

javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated

See full trace in a pastebin.

Looking about this you can find lots of things. Some are a bit mi leading and others give good grasp about what’s happening.

The context here is, we have an edge API that uses other internal API for random access data. We could see this problem for some of the requests (so it was not due a self-signed certificate or anything related). I’m attempting to share my experience of what has happening at the end.

Catching the exception

I started by catching the exception and trying to provide a bit more information about the requests that were producing the exception. I could not find any pattern about the type of requests.

Connection timeout?

One of our sysadmins pointed out that the problem could be generated by the socket timing out during the SSL handshake, as it’s pointed out in the following JIRA ticket: https://issues.apache.org/jira/browse/HTTPCLIENT-1070

While this could be the cause of the problem, we wanted to make sure that this was the case.

Analysing SSL errors

The “How to Analyse Java SSL Errors” post from dzone.com give me a good light about how to proceed. So I turned on the SSL debugging information by adding the following parameter when starting the server and try to find the problem:

-Djavax.net.debug=ssl:handshake

This is what I found:

qtp978883631-84, WRITE: TLSv1 Alert, length = 2
qtp978883631-91, WRITE: TLSv1 Handshake, length = 48
qtp978883631-84, called closeSocket()
qtp978883631-84, IOException in getSession():
    java.net.SocketTimeoutException: Read timed out
qtp978883631-84, called close()
qtp978883631-84, called closeInternal(true)

This is a great indicator that as our Systems guy was suggesting, connection is timing out during the SSL handshake.

At the end

The way I’ve tackled this issue is by silencing this kind of exceptions. We could have tried retrying the connection but we thought that the information provided by the second API is not as important and we will have the chance of trying again later in the following requests.

 

 

 

 

 

Llega Java 8

Que hay de nuevo en Java 8?

Ya llevo tiempo programando en Java y toca recopilar un poco de información para ver que incluye la nueva versión de Java 8.

Esta vez escribo en Español porqué en Inglés ya hay mucha literatura al respecto.

Expresiones Lambda

Es una nueva funcionalidad del lenguaje que nos permite incluir “funcionalidad” a los parámetros de los métodos. Es mas que el “sintactic sugar” (como dicen algunos), pues el compilador puede usar el contexto de la expresión para determinar la interfaz usada y resolver el tipo de los parámetros automaticamente.

Un ejemplo de la frase anterior es lo siguiente:

En vez de:

(String str) -> System.out.println(str)

Podemos hacer:

str -> System.out.println(str)

Metodos “default”

Nos permite añadir funcionalidades en las interfaces. Esto nos permite también añadir nuevos métodos a las interfaces sin romper todas las implementaciones actuales (backwards compatibility).

El comportamiento de este tipo de métodos es el que podemos intuir por su nombre: Cualquier clase que implementa una interfaz pero no sobre-escribe el método, va a adquirir la implementación por defecto.

Nueva colección “Stream”

La interfaz “stream” nos permite realizar operaciones en un estilo funcional sobre stream de elementos. Representa la secuencia de objetos, de un modo similar a un iterador, pero permite su ejecución en paralelo. Permite operaciones típicas como “map”, “reduce” o “filter”.

Un ejemplo de uso de los nuevos “Streams”:

List stringCollection = new ArrayList<>();
stringCollection.add("ddd2");
stringCollection.add("aaa2");
stringCollection.add("bbb1");
stringCollection.add("aaa1");
stringCollection.add("bbb3");
stringCollection.add("ccc");
stringCollection.add("bbb2");
stringCollection.add("ddd1");
 
stringCollection
    .stream()
    .filter((s) -> s.startsWith("a"))
    .forEach(System.out::println);

Programación funcional

Java 8 trata de elevar las funciones en ciudadanos de primera dentro de la programación, justo como se trata en la programación funcional.

Por eso, se incluye la interfaz “Function”, el tipo de datos “Tuple” al mas puro estilo Scala, se añaden algunos tipos inmutable (DateTime, Streams o el Optional, así como ya lo son los String hoy en día).

Como ya hemos dicho, además de las expresiones, también incluyen nuevas interfaces que habilitan Java para un uso mas funcional. Un ejemplo de uso es el siguiente:

Function<String, String> atr = (name) -> {return "@" + name;};

Y como empezar?

Para terminar, podéis descargar Java Runtime Environment. Ahí tenéis un review con un botón al link para descargarlo.

Querying AWS ELB access logs with EMR

On March of 2014 Amazon announced a new feature for ELB (Elastic Load Balancer): Access logs.

Those logs contain access information, and from there you can extract things like the average response time of 3 involved parts:

  • Request processing time: Total time elapsed (in seconds) from the time the load balancer receives the request and sends the request to a registered instance

  • Backend processing time: Total time elapsed (in seconds) from the time the load balancer sends the request to a registered instance and the instance begins sending the response headers.

  • Response processing time: Total time elapsed (in seconds) from the time the load balancer receives the response header from the registered instance and starts sending the response to the client. This processing time includes both queuing time at the load balancer and the connection acquisition time from the load balancer to the backend.

See the full AWS ELB access log documentation.

This blog post is about querying ELB (Elastic Load  Balancer) access logs with EMR (Elastic Map Reduce).

1 – Initializating EMR instance

elastic-mapreduce --create --alive --name "Pau Gay - AWS ELB Logging analysis" --hive-interactive --num-instances 4 --master-instance-type c3.xlarge --slave-instance-type c3.4xlarge
elastic-mapreduce --ssh {jobId}

# once logged into the box
sudo apt-get install tmux -y
tmux
hive

2 – Running the hive command

add jar file:/home/hadoop/.versions/hive-0.8.1/lib/hive-builtins-0.8.1.jar;
add jar file:/home/hadoop/.versions/hive-0.8.1/lib/hive-serde-0.8.1.jar;
add jar file:/home/hadoop/.versions/hive-0.8.1/lib/hive-contrib-0.8.1.jar;

SET hive.mapred.supports.subdirectories=true;
SET mapred.input.dir.recursive=true;

DROP TABLE elb_raw_access_logs;
CREATE EXTERNAL TABLE elb_raw_access_logs (
  `Timestamp` STRING,
  ELBName STRING,
  RequestIP STRING,
  RequestPort STRING,
  BackendIP STRING,
  BackendPort STRING,
  RequestProcessingTime STRING,
  BackendProcessingTime STRING,
  ClientResponseTime STRING,
  ELBResponseCode STRING,
  BackendResponseCode STRING,
  ReceivedBytes STRING,
  SentBytes STRING,
  RequestVerb STRING,
  URL STRING,
  Protocol STRING
)
ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.RegexSerDe'
WITH SERDEPROPERTIES (
  "input.regex" = "([^ ]*) ([^ ]*) ([^ ]*):([0-9]*) ([^ ]*):([0-9]*) ([.0-9]*) ([.0-9]*) ([.0-9]*) (-|[0-9]*) (-|[0-9]*) ([-0-9]*) ([-0-9]*) \"([^ ]*) ([^ ]*) (- |[^ ]*)\"$"
)
LOCATION '{YOUR_ELB_ACCESS_LOGS_S3_LOCATION};

--
# you can test all works with the following query
select * from elb_raw_access_logs limit 10;

# and this will give you the average response time metrics
select
  avg(RequestProcessingTime) as average_request_processing_time,
  avg(BackendProcessingTime) as average_request_processing_time ,
  avg(ClientResponseTime) as average_request_processing_time
from elb_raw_access_logs;

This is all.

I’m sharing this hive query because I ran in lots of troubles with the RegexSerDe, so I figure out myself the addition of the above listed jars. See that you can only map a regular expression to a String, but yo can do casts afterwards to convert data into your preferred type.

Nb. Remember to terminate your EMR instance:

root@pau-vm:~# elastic-mapreduce --list --active
j-ABCD     WAITING        xxxx         Pau Gay - AWS ELB Logging analysis
   COMPLETED      Setup Hive
root@pau-vm:~# elastic-mapreduce --terminate j-ABCD
Terminated job flow j-ABCD

 

Learning Symfony 2

There are some good friends that has recommend me to learn Symfony 2. I’ve been programming with Zend Framework (not even version two) and with our own framework at VisualDNA, so I was eager to see how the frameworks ecosystem has evolved during this 2 or 3 years that I haven’t been keeping myself updated.

I’ve done the first part of the learning stage, which consists in information gathering. In this post I only wanted to aggregate some useful resources I’ve found and that I still need to consume myself.

Reading

This is basically browsing on the Symfony 2 official documentation. It’s pretty well organised and structured.

Not much going on here, really, but I think it’s interesting to spend some time having a first read about what the official documentation says.

Practising

On the “getting started” you have already checkout and played a bit with the code, but there are two good blog posts / tutorials that look interesting:

Doing this tutorials looks really interesting but the real way to learn is to try it yourself without looking at those. Probably, think about something you have wanting to do for some time and just do it using Symfony 2, that will be IMO the best way to learn.

Videos

Finally, there is some good amount of knowledge on the Internet regarding to video material. Talks from the “Symfony Live” in Paris or San Francisco, or many other conferences that has been happening recently. Here they are:

That’s all for me. If you have any useful resource where you can learn from, please, leave a comment and I will be very happy to update this list of resources.

My first Katayuno

Coding Katas are small problems from a known domain, with the objective of practise basic programming elements and TDD.

Katayunos are the combination of “Coding Katas” and “Desayunos” (which means breakfast in Spanish). It consists on sitting in pairs, in front of a laptop, choose one of the available katas and programming language and have breakfast while working on the given problem practising TDD and thinking about the best approach to solve the problem.

Well, this weekend I’ve been involved on my first Katayuno experience, at Softonic offices. The practised kata was KataBankOCR, where we are given with an ASCI numbers and we have to transform that into a String, not very complex but it had an interesting point into the test.

At the first iteration, we were tackling the problem wrong. We spend some time trying to validate the input and where a bit slow because our strict state (RED – GREEN – REFACTOR) of TDD work-flow. At the end of the first iteration we were discussing about how much we have to cover with the test. On that problem, we can’t test much more than “for a given input” which is the “expected output”.

What you think about testing private methods?

Obviously you can’t test private methods straight away, but you can work around that. However, we agree that doing that is not a good test case. The main reason is that we want to test the problem, not our concrete solution about the problem. The point is that at the end of the exercise, we could switch tests and our code shall keep passing both tests.    

On the second iteration we got it right (in PHP) and in the third and last iteration I had the chance to play with Jasmine, a test library for Javascript (pretty cool stuff). I’ve uploaded my code into Github. Pretty good exercise overall, and felt like I’ve learned and enjoyed.

 

How to name interfaces / abstract classes with namespaces

I’m building a new library with the aim to parse venues from Internet (code on Github). Well, aside from the purpose or usage of the library I’m using this project development to try to write the domain oriented code and by the way I’m facing some arquitectural problems.

Basically, in the past the most common convention for naming interfaces or abstract classes was the following:

For the interface (lib/domain/venue/interface.php):

interface venues_domain_venue_interface {}

And for the abstract (lib/domain/venue/abstract.php):

abstract class venues_domain_venue_abstract {}

Now, I want to port this code to PHP 5.3 Namespace format. The translation is not that direct, note that “abstract” and “interface” are reserver keywords and I can not use then to name my clases.

For the interface (lib/Domain/Venue/Interface.php):

use Venues\Domain\Venue;
 
interface Interface {}

And for the abstract (lib/Domain/Venue/Abstract.php):

use Venues\Domain\Venue;
 
abstract class Abstract {}

That will simply break with a PHP Parse Error because the “Interface” or “Abstract” are reserver keywords.

Researching on Internet I’ve found the following options:

 

1. Prepend the domain name

For the interface (lib/Domain/Venue/VenueInterface.php):

use Venues\Domain\Venue;
 
interface VenueInterface {}

And for the abstract (lib/Domain/Venue/VenueAbstract.php):

use Venues\Domain\Venue;
 
abstract class VenueAbstract {}

 

2. Append the domain name

For the interface (lib/Domain/Venue/InterfaceVenue.php):

use Venues\Domain\Venue;
 
interface InterfaceVenue {}

And for the abstract (lib/Domain/Venue/AbstractVenue.php):

use Venues\Domain\Venue;
 
abstract class AbstractVenue {}

 

3. Prepending “I” or “A”

For the interface (lib/Domain/Venue/IVenue.php):

use Venues\Domain\Venue;
 
interface IVenue {}

And for the abstract (lib/Domain/Venue/AVenue.php):

use Venues\Domain\Venue;
 
abstract class AVenue {}

 

4. Append “Class”

For the interface (lib/Domain/Venue/InterfaceClass.php):

use Venues\Domain\Venue;
 
interface InterfaceClass {}

And for the abstract (lib/Domain/Venue/VenueClass.php):

use Venues\Domain\Venue;
 
abstract class VenueClass {}

 

References:

Get Facebook access_token

I’m going to try to document the experience of creating an access_token so my app. can post messages on my Facebook Page. As a summary, I have the following elements:

1) Facebook Page

This is the Doonish Facebook Page. It’s a page that describes the project and that try to get as many fans as possible.

2) Facebook Application

In the other side I’ve created an application for allowing the remote post to my page. This is application that will do that.

What I want to do is promote each day a new question from doonish into my Doonish Facebook Page.

 

How to automatically post in a Facebook Page?

1) Get application “access_token”

The first step is to get application “token_id” so we can post to the Facebook Page.

https://www.facebook.com/dialog/oauth?
    client_id={facebook-app-id}&
    redirect_uri={your-url}&
    scope=manage_pages&
    response_type=token

That will return with something like the following:

access_token={your-access-token}

2) Find the page “access_token”

Then, using the above token, you can go to the following location:

https://graph.facebook.com/me/accounts?
    access_token={your-access-token}

And you will see something like the following:

{
    "data": [
    {
        "name": "{your-page}",
        "access_token": "{page-token-id}",
        "category": "Website",
        "id": "{page-id}"
    },
    ...
    ]
}

3) Write the actual script

Whith the above “page-token-id” you can then write a PHP snippet that can do something like the following:

/**
 * Promote
 *
 * This method will automatically post a message on the facebook wall.
 *
 * @param integer $questionId The id of the question that we want to 
 *     promte.
 */
public function promote($questionId)
{
    $message = $this->generateMessage($questionId);
 
    $facebook = new Facebook(
        array(
            'appId'  => $this->facebookConfig['app_id'],
            'secret' => $this->facebookConfig['secret'],
            'scope'  => $this->facebookConfig['scope'],
            'cookie' => $this->facebookConfig['cookie'],
            'acceptUrl' => $this->facebookConfig['acceptUrl']
        )
    );
 
    $status = $facebook->api(
        '/' . $this->facebookConfig['page_id'] .'/feed',
        'post',
        array(
            'access_token' => $this->facebookConfig['token_id'],
            'message' => $message,
            'cb' => ','
        )
    );
}

Hope this code is useful for someone, it took me a while to figure out the right way to do it. I’ve used Facebook PHP SDK and I’ve got all the information from the Authentication page from Facebook Developers.

Profiling your database in Zend Framework

Just a quick tip for all Zend Framework developers.

Imagine I’m getting an error from a select that I’ve created:

$questionRow = $this-&gt;fetchRow($select);

If you want to see which query is being executed on your database, you can go with something like that:

$db = Zend_Db_Table::getDefaultAdapter();
$profiler = $db-&gt;getProfiler()-&gt;setEnabled(true);
 
// like before
$questionRow = $this-&gt;fetchRow($select);
 
$query = $profiler-&gt;getLastQueryProfile();
var_dump($query);
die;

That will print the SQL query that has just been executed.

doonish: Status update

Llevo un tiempo dedicando esfuerzo a doonish.es, el juego de preguntas y respuestas basado en Trivial Pursuit y ahora me he dado cuenta de que he dejado un poco abandonado el blog.

Tan solo resumir un poco lo que he estado movido entre manos durante estos últimos meses:

1) Trato especial a los usuarios que vienen de Google

Nos dimos cuenta de que nos llegan algunas visitas de Google y antiguamente si el usuario hacía click en nuestra página entraba directamente a la página de la pregunta. Ahora no, ahora si el usuario no tiene cookies (no es un usuario que conocemos) se le muestra directamente la respuesta. Esto lo hacemos porque entendemos que en este caso concreto el usuario quiere conocer la respuesta a la pregunta, así que intentamos satisfacer de algún modo su necesidad.

Ejemplo: ¿Cuál es la capital de Alaska?

Esto es lo que ve un usuario que conocemos:

Esto es lo que ve un usuario que no conocemos y que viene de google:

2) Ya tenemos SEO

Salió sin buscarlo. Me estuve intercambiando algunos correos con Manel Pérez. Aproveché nuestro buen trato y sus grandes conocimiento (trabaja en el departamento de SEO de Softonic) para preguntarle algunas dudas sobre  SEO. Fuimos charlando y al final vimos que había interés por ambas partes de que se incorporara al equipo de doonish. Ya tenéis la información en la página de “Sobre nosotros” de doonish.

3) API

En un tiempo récord he escrito una API para doonish. Es muy sencilla pero permite a desarrolladores acceder a la base de datos de preguntas de doonish. De momento no tengo muy claro como enfocarlo y la he hecho privada. Si a alguien le interesa aquí dejo cinco céntimos sobre mas o menos que endpoinds se han habilitado:

question.json
  • http://doonish.es/api/question.json
question/[CATEGORY].json
  • http://doonish.es/api/question/cine.json
  • http://doonish.es/api/question/corazon-y-famosos.json
categories.json
  • http://doonish.es/api/categories.json
Esto en principio va a permitir a otros desarrolladores jugar con nuestra base de datos. De momento la API es privada por lo que no se puede ver el contenido, adjunto una imagen para que os hagáis una idea.
Y bueno, todo esto lo acompaño con más de 2000 preguntas y ya casi 500 usuarios. Como decimos en Catalán: “mica en mica s’omple la pica” :) !

Configurar l’idioma Català a Linux

Estic fent algunes probes per a mostrar les dates a PHP en Català. Al meu servidor local tot funcionava correctament, pero al passar-ho al servidor remot ha deixat de funcionar.

El codi font es el següent (en PHP):

setlocale(LC_ALL, $lang . "_ES.UTF-8");
echo strftime("%A %e de %B del %Y a " . $translate-&gt;_("las") . " %H:%M", strtotime($time));

En castellà funcionaba correctament, pero al passar-ho al català no funciona.

Per a que funcioni, he hagut de fer la següent modificació: Obrir el fitxer /var/lib/locales/supported.d/local i afegir la darrera línea.

es_ES.UTF-8 UTF-8
ca_ES.UTF-8 UTF-8

Despres de desar el fitxer, he executat la següent comanda:

sudo dpkg-reconfigure locales

I ja ha funcionat, ja puc veure la data tant en català com en castellà :) !