Almost there, we are days away

The last few days of tinkering before the start of the fantasy league normally means a frantic decision making process where the teams change numerous times. I even tried to get Kane, Salah & De Bruyne in the same team, but then the other players I had to compromise too much.

Looking at the stats for ownership it seems Sergio Agüero is the one most people are opting for. Since the Community Shield his ownership has more than doubled, whilst the others like Firmino are losing ownership.

I will spend the next few days analysing the data and fixtures before I finally lock in my selection for the first game.

Back to the stats and hopefully a clear definition of who I need will jump out at me.

 

 

Almost here

Small posts as we enter the final week before the start of the fantasy league.

We are into the final few days before the season kicks off and I’m still tinkering. I’m trying to find that jewel that no one else has found. Currently the bargains are to be found with the teams that have been promoted. Two of the teams have invested heavily and the selections are swaying towards those teams.

 

The top three selected players from the promoted teams are:

  1. da Silva Neves-Rúben Diogo 14.40% Ownership
  2. Sessegnon-Ryan 11.30% Ownership
  3. Kamara-Aboubakar 10.90% Ownership

The top selected player from Cardiff is Connolly-Matthew and he has been selected by 1.8%

I think Fulham and Wolves can be top 10 contenders where Cardiff will struggle.

Away from the promoted team the player with the most movement this week is Sergio Agüero he has increased by 4.9% Ownership, its amazing what a couple of goals can do.

Currently the top 5 selected players are:

  1. Salah 52%
  2. Firmino 35.20%
  3. Kane 32.8%%
  4. Zaha 31.30%
  5. De Gea 29.40%

Interestingly Courtois has 5.90% Ownership and he is not training with Chelsea at the moment, I think he may be leaving.

 

More stats will be coming out and expect more dashboards.

 

 

 

 

 

Less than 2 weeks to go

The planning stage is almost over, the games are just around the corner the fun is about to begin. Over the coming weeks the new dashboards will be done and hopefully this season they will be bigger and better then ever. I’ve been working on some new stats to see if I can pick a better team to start with. The last few seasons I have used my wildcard in the first few weeks, mind you I won a few mini leagues last season so it wasn’t all bad.

My Captains pick was the thing that was good last season, and this was one of the reasons I did win some of the mini leagues I entered.

So.…..What are  my tactics for this season?

  1. Pick a spine for my team made up of players that finished in the top 4 last season. This means expensive picks.
  2. Look at players who play in a different position to what they are shown in the fantasy league, so a defender that plays more forward.
  3. look at the new signings. Salah was fantastic last season 🙂
  4. Pick players for the bench that actually play.
  5. Look at the first six fixtures for the promoted teams to see if I can pick a bargain.

If we look at the promoted teams and who they are playing, my predictions are in brackets

Cardiff have:

  • Away Bournemouth (D)
  • Home Newcastle (W)
  • Away Huddersfield (D)
  • Home Arsenal (L)
  • Away Chelsea (L)
  • Home Man City (L)

Fulham have:

  • Home Crystal Palace (D)
  • Away Spurs  (D)
  • Home Burnley (W)
  • Away Brighton (W)
  • Away Man city (L)
  • Home Watford (W)

Wolves have:

  • Home Everton (W)
  • Away Leicester (D)
  • Home Man City (L)
  • Away West Ham (D)
  • Home Burnley (W)
  • Away Man United (D)

Based on this I believe there could be bargains to have, but this is not guaranteed. In my initial 15 man squad I have 1 player from each promoted team.

Looking at all the teams I have a few selection headaches. Hopefully my Dashboards will help 🙂 Here are some of my headaches.

  • De Bruyne or Sane or Sterling or Mahrez?
  • Pickford or not for the main goalkeeper?
  • Kane or salah as my big name signing? maybe both but this would mean almost a quarter of the budget on 2 players.
  • Is it worth going for Arsenal players in the post Wenger era.

Its going to be a tense few weeks leading up to the kick off, but it will be fun.

 

2018/2019

The World Cup is over but we don’t have to wait long for the Premiership. The Fantasy League is already live and there will be some interesting choices as the transfer rumours turns into actual transfers, this also means the data needs to start crunching 🙂 I have done some initial data loads and I’m starting to look at the data to see what it tells me, I will keep you informed. I will be looking at the various position and comparing average values with last season to see what improvements players have had or where they have fell off.

One big decision is “Do you include Salah?”

His price has gone up by a massive amount, but I suspect most teams will include him, so tactically he may be one of those where you forget the price and include him. This will be one of those decisions that could make or break a season.

Anyway its back to the data crunching and see if I can pick a winning team.

Let the good times begin 🙂

Transfers and remaining games

Over the last few months I have been creating Fantasy Football data flows using Azure which allows me to automate the data flows. This has enabled me to start looking at what data is important. I’ve been concentrating on the Transfers just lately. So here is a Power BI dashboard which shows almost real time data (Data captured every 15 minutes)

In this dashboard I have the ability to look at various pieces of data:

  • Transfer data. This allows me to see how big a player is expected to be in the coming game week.
  • Goals and assists for those players. I can select the game weeks so I can select what I believe is the right amount of weeks to go back over, 3 to 6 appears to be the ‘good zone’.
  • Remaining games details.

 

The actual Transfer section starts off at the team level then you drill down to players.

I love the capabilities of Power BI to drill down like this, it really gives you some options.

 

 

The whole dashboard becomes the selection options. If I actually select Teams or Positions only the data which meets that criteria will be shown, if I select something on the charts the rest stays there but looks opaque. Here I have selected Harry Kane as he wasn’t high up the charts for scoring/Assists.

The final part allows me to see the fixtures. Now this could be an interesting selection to highlight. Over the years I have observed that the first 6 and last 6 games of the season throws up the biggest surprises. Take this season, West Brom have beaten Man united and drew with Liverpool in the last few weeks, and they are in the relegation zone.  Spurs have West Brom so I suspect that the players who are bringing in Harry Kane expect a big return, but could there be surprise.

I still have a lot of data to blend to get me to where I want to be, but I’m building up some good history, this at least gives me a good place to start.

Having the historical data allows me to change calculations and then apply the new settings to see what happens.

Roll on next season 🙂

 

 

 

Real Time Data Collecting – ver2

In the last post I explained how I collect the data using Azure by means of Webjobs, event hubs etc, there is a simpler version. Python.

I’m still using Azure but this time I have a VM which has a series of Python scripts. These scripts are run at different times based on the requirements. The actual Architecture diagram looks like this.

Basically, I use Windows scheduler to run different Python scripts at various times.

  • 5 minutes interval – These are used for up-to-date transfers of fantasy premier League players. The scripts extract the information from the json files and inserts it into a SQL db. My on prem SQL db runs a stored procedure  at a predefined time which moves the data across then deletes the old records from the cloud. This enables me to keep the db in Azure at a decent size.
  • Weekly interval – These are used to get the data based on points, minutes played, cards received etc. The new data is pulled across to an on prem db in the same way as the transfer data.

The Python scripts works in the following way.

  • A main script is run via the schedule, this calls the specific Python files that reads the data. The DB connections are held in another Python file. This way I have one connection file which the others ‘IMPORTS’

We are at the tail end of the season, so all these scripts will be sued in ‘Anger’ next season. I have been using them mind you and I have found some good insights.

 

My team for wk34 which is a double game week I used some of this data to pick my players.

  • Captain – Eriksen: 2 games and he has been doing well recently. So far he has scored 1 goal, so I should expect a decent return.
  • GK – Schmeichel: 2 games so I took out WBA keeper for him, my other choice was Pickford so against Man Utd I would have never thought they would keep a clean sheet. Still 1 game to go for Kasper so I might get something. His points were slightly swayed by his penalty save.

My other players were my stalwarts which have been in there for a long time, mind you the City players didn’t return much, now they are champions maybe time for a change.

It’s an interesting time of the season for a few reasons.

  • Premiership race is over, so it’s the next three places that are up for grabs, so Spurs, Liverpool, Man Utd and Chelsea players have it all to play for.
  • Relegation is still not decided so teams will be fighting for their Premiership survival and expect unusual results Like Man Utd getting beat at Home by WBA.

I need to look at the fixtures and the stats together next. Next season this will be an automated process. I’m hoping to click on a button and it will give me a narrative on the coming weeks 🙂

Loads more to do really, the thing is the more data I look at the more data I want. Its like going round in circles 🙂

 

Real time data collecting

This is the first part in my series on collecting data

I thought I would go back to the beginning and post about how I collect the data, as the reports are nothing without data. Most of the charts are based on data which is updated once a week, things like Goals or minutes played so the actual data capture is relatively simple, but for some of the data like transfers this gets updated throughout the week so I wanted to see what insights I could get from that. When I say transfers I’m referring to the way people move players in and out of their teams every week.

Collecting real-time data is something I have experience with as I designed a process for work, so I use the same building blocks. The overall design looks like this, but lets break it down.

The APIs I use are provided for anyone, so its just a case of having the knowledge on how to use them.

A Webjobs – runs continuously and gets the data, the data is too big to be sent to an Event Hub so I do some work on the data, I extract the nodes I want and send that to an even Hub but I also send the entire json file to a Storage account. The reason being I can use that data when I need it.

The data that is sent to the Event Hub is consumed by the Stream Analytics and ends up in an Azure SQL database. Within the Stream I can modify the data or add additional data if I so wish.

In my design I also want to merge this data with other data sources so I use Azure Data Factory to copy the data from the cloud to a SQL db on a local machine which is being used as a Data Warehouse and contains all the other football data. I have various processes running which collects different types of data.

At this point I have real football data and fantasy football together so I can start to blend it together to see what insights I can get.

This is an ongoing project for me which I have been working on as a side project for many years in which time I have changed and modified all the elements. Originally I used VB.net to write stuff but now I use C#, I removed ETLs which were based on SSIS and now I use Azure or even Python, cant wait till the support for Python in Function Apps is better 🙂

So that was a brief overview but it gives you the main parts and how I collect from that data source.

 

Next part will be based on the other data sources and how we blend data before visualizing it.

 

Latest dashboards

Been playing again with Power BI, the latest dashboards shows various stats to assist with my player picking:

Points & Average points by teams and positions  over a given time period.

Points & Average points by players over a given time period.

Points & Average points by players outside the top 6 teams  over a given time period.

Goals scored & Assists by teams and positions  over a given time period.

Goals scored & Assists by players  outside the top 6 teams  over a given time period.

If I drill down on the assists to show players you can see Benteke had 4 assists prior to last weeks games, who’d have thought that 🙂

 

Goalkeepers stats.

The purpose of these dashboards is to show what is happening and to give you the ability to ‘drill down’ from teams to players. Need to start looking at the next game week and think about game week 31 as that is a reduced game week in terms of games.

 

More APIs for Fantasy data.

After a bit of digging I found some useful APIs to get data, so I created some more Python scripts to capture data and now I’m in a position where I’m looking at that data and considering redesigning the entire data model. The reason for this is some of the data I was creating will be more accurate with the new APIs.

The new data allows me to summarise the players data by week. I have made a few more dashboards to show the data so I can see what I can do. Here they are.

This gives me a summary based on the last 6 weeks.  I can see things like minutes played and goals. Here I have selected Midfield.

If I select Hazard from the treemap I get his stats. This will enable me to see how his points are broken down.

And the big question is normally Kane or Aguero for my forward.

Based on the last 6 weeks Sergio wins 🙂 but what happens if we look at the last 3 weeks, well, Firmino comes into the mix

The good thing is we are getting to a point where the data is enabling us to make a decision. With footy, emotions play a major part so having decisions based on data rather than emotions may make a different.

What happens if we want to include price. We can add Price and Price range to the dataset. This will give us the ability to look at players within a certain budget. Having the ability to select the weeks you want to look at will also show how they are progressing or not!

Adding in a price range we can select our budget. I want to look at the Midfield players in the £9 million to £11 million range, as expected the big hitters like Salah and KDB are there. In the last three weeks they have amassed 9 goals and 7 assists.

 

If I select KDB I can see that he has scored once and has 4 assists. The bottom right hand chart shows me he has had a price increase as well

But if my budget was less, in the £5 million to £7 million range it would look like this

The top two being Prowse and Ramsey, so lets look at them. 1 goal 2 assists for Prowse

3 goals for Ramsey in a single game, and his game time is less.

So in this situation I would probably pick Prowse, and he is cheaper.

In summary leveraging Power BI to visualise the data will hopefully give me more of a competitive edge going forward 🙂 Remember spend time on getting the data right then the Power BI part is easier

Next I need to use Python to provide some stats prior to displaying in Power BI.

 

Home Team dashboard

I’ve spent more time playing on dashboards, it appears the more time I spend the more options I want to include :). I decided to go with a Home Team dashboard.

You pick a League and a season and it gives you the home info.

If you select a Team you get the details

If we look at the games in details we can see that home draws are going to cost Liverpool.

You can look at the data across multiple season as well, here I selected 3 seasons and it gave me this data.

I’ve built quite a few dashboards now, I need to build the away dashboard next then I’m onto the predictive analysis stuff. I will then start to use R/Python to really push the data. The current Predictions stuff is not accurate enough yet so I’m busy building in some other variables.

Its amazing what you can add in and luckily with some of the variables I can retest all the previous results to see what effect it has. Some of the variables I cant use as I only have certain data points but going forward that will grow.

During my time on this side project it has made me realise the importance of collecting data, but not just collecting it but actually using it.