Skip to main content

Tashkent House Price Analysis


Recently  I've got an idea to analyze prices of houses in Tashkent city.
"www.zor.uz" is one of the most famous sites where people sell houses, cars, electronics and etc.
I had to crawl it and collect data from it.
Unfortunately, "www.zor.uz" does not have the old data, the latest data that  I found there was just one month old. They seem to clean their DB every month.

I needed data for previous years. So, I used "Way Back Machine" that has captured points of all website around the world. It is really a cool stuff to try : https://archive.org/index.php
But they use "https" so I had download their certificate and register it to my JRE.

I crawled that site and succeeded to get data starting from 2009.08 (that was the earliest capture point of www.zor.uz)
Interestingly "WayBackMachine" captures websites periodically, depending how often website changes. They have their own logic to capture sites for optimizing their storage.

Anyways, The data I crawled was from specific months of the year.
After crawling the data, for simplicity I just exported data to Excel sheet.
Then filtered data (removed duplicates, meaningless entities, spams, and zero values) and sorted by date.

Here is graph that displays price change for houses with 2 rooms:


You can see that average prices of 2-room houses during 2009.07 ~2014.08  raised from 30,000$ to 42,000$

By drawing the moving average we can identify when the prices was raising and falling:


Interestingly, houses prices raise during summer and it drops during the winter.
There can be more results can retrieved, but at the moment I am too busy with other projects, and will upload more stats later on.



Comments

Popular posts from this blog

NLP for Uzbek language

    Natural language processing is an essential tool for text mining in data analysis field. In this post, I want to share my approach in developing stemmer for Uzbek language.      Uzbek language is spoken by 27 million people  around the world and there are a lot of textual materials in internet in uzbek language and it is growing. As I was doing my weekend project " FlipUz " (which is news aggregator for Uzbek news sites) I stumbled on a problem of automatic tagging news into different categories. As this requires a good NLP library, I was not able to find one for Uzbek language. That is how I got a motive to develop a stemmer for Uzbek language.       In short,  Stemming  is an algorithm to remove meaningless suffixes at the end, thus showing the core part of the word. For example: rabbits -> rabbit. As Uzbek language is similar to Turkish, I was curious if there is stemmer for Turkish. And I found this: Turkish St...

Three essential things to do while building Hadoop environment

Last year I setup Hadoop environment by using Cloudera manager. (Basically I followed this video tutorial :  http://www.youtube.com/watch?v=CobVqNMiqww ) I used CDH4(cloudera hadoop)  that included HDFS, MapReduce, Hive, ZooKeeper HBase, Flume and other essential components. It also included YARN (MapReduce 2) but it was not stable so I used MapReduce instead. I installed CDH4 on 10 centos nodes, and I set the Flume to collect twitter data, and by using "crontab" I scheduled the indexing the twitter data in Hive. Anyways, I want to share some of my experiences  and challenges that I faced. First, let me give some problem solutions that everyone must had faced while using Hadoop. 1. vm.swappiness warning on hadoop nodes It is easy to get rid of this warning by just simply running this shell command on nodes: >sysctl -w vm.swappiness=0 More details are written on cloudera's site 2. Make sure to synchronize time on all nodes (otherwise it will give error on n...

Why Uzbekistan needs its own local CDN

 Introduction Imagine that you're serving a website and the majority of your users are people from Uzbekistan. In other words, your business is targeting the local market of Uzbekistan.  To make your website faster you will need a CDN, this can help your business to perform better. There are several reasons why your website can be slow without the CDN acceleration: 1. No existing Tier 2 network. Tier 2 network plays an important role when it comes to the speed of the internet. It enables Tier 3 internet service providers to directly connect to the internet without other intermediate layer. In Uzbekistan, UzTelecom is the largest internet provider. According to the ` traceroute ` command it uses RETN tier-2 network. The RETN unfortunetly does not have the lines(network) in  Uzbekistan according to their map ( source ). This means that the majority of internet traffic needs to go through the single UzTelecom, which creates an overhead for the speed of internet. 2. Slow inte...