My Technical Blog

Intro Bloom filter is a probabilistic data structure for searching element in a data set. It is similar to HashSet, similarly it tells us whether the set contains certain element or not. Difference is the output of contains(element)=TRUE is futuristic. In our example we set futuristic value to 0.01 , which means the answer "It contains" is 99% correct. Read more about Bloom filter from here: https://en.wikipedia.org/wiki/Bloom_filter Scenario We create two Arrays of random elements. Elements count in each array is 1,000,000. Then we insert the first array into BloomFilter, and we iterate the first array and check if the item contains in BloomFilter. Second array is used only for checking non-existing elements. We do the same for HashSet as described above. Benchmarking code We used customized version of Bloom filter  which can accept byte array. (Previous version of this blog was using encoding of string for every put and contains, which was misguiding...

    Natural language processing is an essential tool for text mining in data analysis field. In this post, I want to share my approach in developing stemmer for Uzbek language.      Uzbek language is spoken by 27 million people  around the world and there are a lot of textual materials in internet in uzbek language and it is growing. As I was doing my weekend project " FlipUz " (which is news aggregator for Uzbek news sites) I stumbled on a problem of automatic tagging news into different categories. As this requires a good NLP library, I was not able to find one for Uzbek language. That is how I got a motive to develop a stemmer for Uzbek language.       In short,  Stemming  is an algorithm to remove meaningless suffixes at the end, thus showing the core part of the word. For example: rabbits -> rabbit. As Uzbek language is similar to Turkish, I was curious if there is stemmer for Turkish. And I found this: Turkish St...

Docker Docker offical webiste: https://www.docker.com/ Setup Docker Prepare fresh version of CentOS, I am using CentOS 6.7. Update the yum rep. > rpm -iUvh http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm > yum update -y   Install Docker > yum -y install docker-io Pull some image of container, I am going to use CentOS container. To pull the latest (CentOs 7) >  docker pull centos Or > docker pull centos:centos6 Check which container images are installed: > docker images REPOSITORY          TAG                 IMAGE ID            CREATED             VIRTUAL SIZE centos              centos6    ...

Many of us used BitTorrent(or uTorrent) to download files on internet in a short time. Their download speed is high due to Peer-to-peer technology. That means, rather than downloading file from server, we are getting the file from another computer. But how two computers that have a local IP and are behind NAT, how they can connect each other? For that, NAT Traversal methodologies come for help. Note that there are mainly 2 types of NAT: Symmetrical(complex NATs:carrier-grade NAT) and Full (home network or small enterprises). let us consider Full NATs first. Methodologies of NAT traversal are: UPnP - old and hardware oriented method NAT-PMP (later succeeded by PCP)- introduced by Apple, also hardware oriented(i.e: not all routers have it, and even if it had, it is turned off by default) UDP Punching  - this is done by STUN which uses public server to discover NAT public IP & port TCP Punching -  similar to UDP punching but more complicated Symmetrical NATs are...

Recently, I have been interested in sniffing Http requests/responses which are passing through 80/8080 port. This is helpful when you want to trace if there are some data outgoing from your PC without your knowledge. Sniffing(or monitoring) is a popular way to observe http requests. Anyways, I decided to choose Python for programming, because it is easier, portable, and extensible. There are tons of libraries for Python. The next library that i want to introduce is called "scapy" You install it like this: >pip install scapy Then, here is the simple HTTP sniffer: #!/usr/bin/python from scapy.all import * def http_header(packet):         http_packet=str(packet)         if http_packet.find('GET'):                 return GET_print(packet) def GET_print(packet1):     ret = "***************************************GET PACKET****************************************************\n" ...

CDNetworks holds a strong position in Japan's market as we see in the trend analysis chart. Google trends Link MaxCDN is growing quickly due to its open source modifiable Bootstrap UI for users. If we compare MaxCDN with CDNetworks, we can see it here:

How to extend filesystem capacity in hadoop data node? First, attach your new hard drive to your PC (or add virtual hard drive if you are running on VirtualBox) Mount gParted ISO on your CentOS node. Boot with gParted. Create partition table for new hard drive: Device > Create Partition Table>  GPT is recommended Create new partition: "lvm2 pv" is recommended Boot CentOs Stop hadoop services on node. fdisk -l vgextend vg_hadoop2 /dev/sdb1 lvdisplay lvextend -L+8.48G /dev/vg_hadoop2/lv_root resize2fs -F /dev/vg_hadoop2/lv_root df -h That's it