Mark Needham

Thoughts on Software Development

neo4j/cypher: SQL style GROUP BY functionality

with 8 comments

As I mentioned in a previous post I’ve been playing around with some football related data over the last few days and one query I ran (using cypher) was to find all the players who’ve been sent off this season in the Premiership.

The model in the graph around sending offs looks like this:

Sending offs

My initial query looked like this:

START player = node:players('name:*')
MATCH player-[:sent_off_in]-game-[:in_month]-month
RETURN player.name, month.name

First we get the names of all the players which are stored in an index and then we follow relationships to the games they were sent off in and then find which months those games were played in.

That query returns:

+----------------------------+
| player.name  | month.name  |
+----------------------------+
| "Jenkinson"  | "February"  |
| "Chico"      | "September" |
| "Odemwingie" | "September" |
| "Agger"      | "August"    |
| "Cole"       | "December"  |
| "Whitehead"  | "August"    |
...
+----------------------------+

I thought it’d be interesting to see how many sending offs there were in each month which we’d achieve in SQL by making use of a GROUP BY.

cypher has a bunch of aggregation functions which allow us to achieve the same outcome.

In our case we want to use the COUNT function and we want our grouping key to be the month of the year so we need to include that as part of our RETURN statement as well:

START player = node:players('name:*')
MATCH player-[:sent_off_in]-game-[:in_month]-month
RETURN COUNT(player.name) AS numberOfReds, month.name
ORDER BY numberOfReds DESC

which returns:

+----------------------------+
| numberOfReds | month.name  |
+----------------------------+
| 7            | "October"   |
| 6            | "December"  |
| 4            | "September" |
| 4            | "November"  |
| 3            | "August"    |
| 2            | "January"   |
| 2            | "February"  |
+----------------------------+

As far as I can tell anything which isn’t an aggregate function is used as part of the grouping key which means we could include more than one field in our grouping key.

This isn’t particularly relevant for us for this particular query but would become useful if we add the teams that the players play for.

I extended the graph to included a player’s statistics for each game which also includes a relationship indicating which team they played for in a specific game.

The model now looks like this:

Stats

It does now look quite a bit more complicated but this was the best way I could think of modelling player specific details for a match.

I couldn’t see another way of modelling the fact that a player played for a certain team in a match which I want to use for some other queries but if you can see a simpler way please let me know.

To get a list of the red cards and the name of the team the offender played for we can write the following query:

START player = node:players('name:*')
MATCH player-[:sent_off_in]-game-[:in_month]-month, 
      game-[:in_match]-stats-[:stats]-player, 
      stats-[:played_for]-team
RETURN player.name, month.name, team.name
ORDER BY month.name

The original query traversed a path from a player to games they were sent off in and then from the games to the month the game was played in.

We’ve now added a traversal from the game to the game stats for that player and we also traverse from the game stats to the team node that the player played for in that game.

When we run this we get the following results:

+--------------------------------------------+
| player.name  | month.name  | team.name     |
+--------------------------------------------+
| "Agger"      | "August"    | "Liverpool"   |
| "Whitehead"  | "August"    | "Stoke"       |
...
| "Shotton"    | "December"  | "Stoke"       |
| "Nzonzi"     | "December"  | "Stoke"       |
| "Jenkinson"  | "February"  | "Arsenal"     |
...
| "Ivanovic"   | "October"   | "Chelsea"     |
| "Torres"     | "October"   | "Chelsea"     |
+--------------------------------------------+

So we can see that Stoke got 2 players sent off in December and Chelsea got 2 sent off in October.

We can write the following query to return a result set which uses team and month as the grouping key i.e. we count how many paths there are which have the same team and month:

START player = node:players('name:*')
MATCH player-[:sent_off_in]-game-[:in_month]-month, 
      game-[:in_match]-stats-[:stats]-player, 
      stats-[:played_for]-team
RETURN month.name, team.name, COUNT(player.name) AS numberOfReds
ORDER BY numberOfReds DESC

When we run that query we see the following results:

+--------------------------------------------+
| month.name  | team.name     | numberOfReds |
+--------------------------------------------+
| "December"  | "Stoke"       | 2            |
| "October"   | "Chelsea"     | 2            |
...
| "August"    | "Stoke"       | 1            |
| "November"  | "Tottenham"   | 1            |
| "December"  | "Everton"     | 1            |
+--------------------------------------------+

This is all explained in more detail in the documentation but I thought it’d be interesting to write about it from the perspective of someone more used to writing SQL and trying to work out how to achieve the same thing in cypher.

Written by Mark Needham

February 17th, 2013 at 9:05 pm

Posted in neo4j

Tagged with ,

  • http://andypalmer.com Andy Palmer

    You could have a node (I’ve provisionally called it Played in my mind) that the player relates to, which has a :for relation to a team and an :in relation to a game.

    player-[:played]->()-[:for]->team
    player-[:played]->()-[:in]->game

    The node could also be called Position (although footballers tend to keep the same position throughout their careers, so that’s more a property of the player than the match)

    You could also put your stats node in this place
    player-[:played]->like_this-[:for]->team, [:in]->game

  • Pingback: Geek Reading February 18, 2013 | Regular Geek

  • http://www.markhneedham.com/blog Mark Needham

    @andypalmer:disqus Am I right in saying that our two solutions to the problem are pretty similar?

    Having said that your naming makes much more sense than mine and of course reads much more cleanly as well!

  • http://andypalmer.com Andy Palmer

    Yes, our solutions are similar (I tweaked yours, rather than think about it from scratch)

    I would break the direct :sent_off_in relationship from player to game, and have that as a transitive relationship through the game stats (which simplifies the graph a little)

  • http://www.markhneedham.com/blog Mark Needham

    Andy Palmer ah cool.

    I tweaked some of the other queries to see what they’d look like with the naming that you suggested and it reads pretty well:

    # Get all the players who have scored for Arsenal this season
    START team = node:teams(‘name:”Arsenal”‘)
    MATCH team-[:home_team|away_team]-game-[s:scored_in]-player
    WHERE player-[:played]-()-[:for]-team
    RETURN player.name, COUNT(s) as goals
    ORDER BY goals DESC

    # Get all the players that have started for Liverpool this season
    START team = node:teams(‘name:”Liverpool”‘)
    MATCH p=team-[:home_team|away_team]-game-[:played_in]-player
    WHERE player-[:played]-()-[:for]-team
    RETURN player.name, COUNT(player.name) as games
    ORDER BY games desc

  • Pingback: neo4j/cypher: Combining COUNT and COLLECT in one query at Mark Needham

  • Pingback: neo4j/cypher: SQL style GROUP BY WITH LIMIT query at Mark Needham

  • Pingback: neo4j/cypher: Returning a row with zero count when no relationship exists at Mark Needham