Cameron Fletcher

Sentence and Word Analysis #2

This as part two of my post on sentence and word analysis. In part one I discussed my motives for analysing the RSS feed in question. In this post I shall be building upon my initial findings and presenting the C# and SQL code that I used to do so.

I have continued to run the RSS reader periodically and now have 284 job descriptions to analyse. I have run through the initial results and identified the words and sentences that are irrelevant and placed these into a keywords table so that I may strip them from my results. This was quite a lengthy process as there were a significant number of these to exclude - nearly a thousand. Following that, I looked through the results and because of the different permutations of the keywords that I was looking for it was evident that I would need to look within the top 100 words/phrases to identify the ones that I was interested in. I made a decision to leave in keywords that related to job skills in addition to computer languages.

The top 100 keyword/skills results from analysis of 284 job descriptions. The analysis took 9.5 minutes to run.

A link to a backup of the database may be found here: jobs.zip (392.41 kb)
You will need to restore this into SQL Server before the .NET code (below) will work.

A link to the .NET code (C#) is here: RssReader.zip (3.48 kb)
You will need to modify the App.Config file to point to your RSS feed and database.

To run the analysis on the sentances in the database you'll need to execute the 'analyse' stored procedure. Once that has finished execuiting you'll need to perform a select from the 'analysis_results' view to view the results.

Posted: Jun 09 2009, 14:21 by flet0496 | Comments (1) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: .NET

Sentence and Word Analysis #1

I was not put forward for a job recently because I had Windows Forms experience and not WinForms experience stated on my CV. The same agency also said I had not been singled out as they were looking for someone who had worked with MVC, unlike me as I had only worked with the Model View Controller framework.

I have come to understand that I have to not only write my CV to appeal to prospective employers who should know what I’m talking about but also for the multiple layers of incompetent individuals though which my CV must make its journey before arriving before someone who can actually read. I have come to refer to these individuals as the jam layer because whilst I am the icing on the cake (and most of the time, the cake itself - that is, after all, what I get paid for) they are the jam in-between because they have jam for brains.

So, I decided to analyse the results from a jobserve.com search for .NET roles based in the UK and identify the top used keywords and phrases so that I could litter my CV with them in the hope that someone with jam for brains would identify the correlation, even if they don't understand what that means.

To achieve this completed the following steps:

1. I wrote an RSS reader in C# that read my RSS feed (for .NET jobs based in UK) that I'd set up through jobserve.com.
2. The RSS reader then iterated through each posting and called a stored procedure in my SQL database that added the content of the job posting to a sentences table as a string.
3. I then had a stored procedure that split the string into words and added them to a words table along with details of the sentence they were in and their position within that sentence.
4. The more complex bit was looping through the words in each sentence concatenating from one to ten consecutive words together throughout the sentence and placing the resultant string into an analysis table.
5. I then performed a simple groupby query eliminating the conjunctives (and, or, as, etc.) to retrieve the results.

This took me a couple of hours and as jobserve.com only allows you to receive the last 24 hours worth of job postings via RSS I present you with the top 30 ranked words/phrases from 75 job postings (below). The rank column details the number of occurrences of that word within the 75 postings.

Annoyingly, these results are probably more use to anyone wishing to write an appealing CV profile about themselves. For my purposes I probably need a larger dataset to work on – I reckon about 500-600 job postings and more granular analysis of wordsets ie. group words into technical, competency based, business area, etc. – and also to look into maybe the top 100 from specific wordsets rather than just the top 30 generic words.

Part two of this post may be found here.

Posted: May 30 2009, 11:59 by flet0496 | Comments (2) |

• Currently 0/5 Stars.
• 1
• 2
• 3
• 4
• 5

Filed under: .NET