Jack Jack Attack!2024-03-19T13:41:01+00:00https://xjlin0.github.ioJackxjlin0@gmail.comCIDR notation2022-03-20T00:00:00+00:00https://xjlin0.github.io/tech/2022/03/20/CIDR-notation
<h3 id="need-to-assign-ipv4-address-in-a-range-">Need to assign IPV4 address in a range? <img src="/assets/imgs/change_math.png" alt="Various Edna Mode Styles" width="40%" /></h3>
<p>Here is just a CIDR(<a href="https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing">Classless Inter-Domain Routing</a>) table for my reference on practicing GCP VPC.</p>
<hr />
<table cellspacing="0" border="0" style="width: 100%; position: relative; border-collapse: collapse;">
<thead>
<tr>
<th style="background: white; position: sticky; top: 0;">CIDR prefix length</th>
<th style="background: white; position: sticky; top: 0;">Dotted Decimal Netmask</th>
<th style="background: white; position: sticky; top: 0;">Number of Classful Networks</th>
<th style="background: white; position: sticky; top: 0;">Number of Usable IPs</th>
<th style="background: white; position: sticky; top: 0;">Note</th>
</tr>
</thead>
<tbody>
<tr>
<td>/1</td>
<td>128.0.0.0</td>
<td>128 As</td>
<td>2147483646</td>
<td>2^(32-1)-2</td>
</tr>
<tr>
<td>/2</td>
<td>192.0.0.0</td>
<td>64 As</td>
<td>1073741822</td>
<td>2^(32-2)-2</td>
</tr>
<tr>
<td>/3</td>
<td>224.0.0.0</td>
<td>32 As</td>
<td>536870910</td>
<td>2^(32-3)-2</td>
</tr>
<tr>
<td>/4</td>
<td>240.0.0.0</td>
<td>16 As</td>
<td>268435454</td>
<td>2^(32-4)-2</td>
</tr>
<tr>
<td>/5</td>
<td>248.0.0.0</td>
<td>8 As</td>
<td>134217726</td>
<td>2^(32-5)-2</td>
</tr>
<tr>
<td>/6</td>
<td>252.0.0.0</td>
<td>4 As</td>
<td>67108862</td>
<td>2^(32-6)-2</td>
</tr>
<tr>
<td>/7</td>
<td>254.0.0.0</td>
<td>2 As</td>
<td>33554430</td>
<td>2^(32-7)-2</td>
</tr>
<tr>
<td>/8</td>
<td>255.0.0.0</td>
<td>1 A or 256 Bs</td>
<td>16777214</td>
<td>2^(32-8)-2</td>
</tr>
<tr>
<td>/9</td>
<td>255.128.0.0</td>
<td>128 Bs</td>
<td>8388606</td>
<td>2^(32-9)-2</td>
</tr>
<tr>
<td>/10</td>
<td>255.192.0.0</td>
<td>64 Bs</td>
<td>4194302</td>
<td>2^(32-10)-2</td>
</tr>
<tr>
<td>/11</td>
<td>255.224.0.0</td>
<td>32 Bs</td>
<td>2097150</td>
<td>2^(32-11)-2</td>
</tr>
<tr>
<td>/12</td>
<td>255.240.0.0</td>
<td>16 Bs</td>
<td>1048574</td>
<td>2^(32-12)-2</td>
</tr>
<tr>
<td>/13</td>
<td>255.248.0.0</td>
<td>8 Bs</td>
<td>524286</td>
<td>2^(32-13)-2</td>
</tr>
<tr>
<td>/14</td>
<td>255.252.0.0</td>
<td>4 Bs</td>
<td>262142</td>
<td>2^(32-14)-2</td>
</tr>
<tr>
<td>/15</td>
<td>255.254.0.0</td>
<td>2 Bs</td>
<td>131070</td>
<td>2^(32-15)-2</td>
</tr>
<tr>
<td>/16</td>
<td>255.255.0.0</td>
<td>1 B or 256 Cs</td>
<td>65534</td>
<td>2^(32-16)-2</td>
</tr>
<tr>
<td>/17</td>
<td>255.255.128.0</td>
<td>128 Cs</td>
<td>32766</td>
<td>2^(32-17)-2</td>
</tr>
<tr>
<td>/18</td>
<td>255.255.192.0</td>
<td>64 Cs</td>
<td>16382</td>
<td>2^(32-18)-2</td>
</tr>
<tr>
<td>/19</td>
<td>255.255.224.0</td>
<td>32 Cs</td>
<td>8190</td>
<td>2^(32-19)-2</td>
</tr>
<tr>
<td>/20</td>
<td>255.255.240.0</td>
<td>16 Cs</td>
<td>4094</td>
<td>2^(32-20)-2</td>
</tr>
<tr>
<td>/21</td>
<td>255.255.248.0</td>
<td>8 Cs</td>
<td>2046</td>
<td>2^(32-21)-2</td>
</tr>
<tr>
<td>/22</td>
<td>255.255.252.0</td>
<td>4 Cs</td>
<td>1022</td>
<td>2^(32-22)-2</td>
</tr>
<tr>
<td>/23</td>
<td>255.255.254.0</td>
<td>2 Cs</td>
<td>510</td>
<td>2^(32-23)-2</td>
</tr>
<tr>
<td>/24</td>
<td>255.255.255.0</td>
<td>1 C</td>
<td>254</td>
<td>2^(32-24)-2</td>
</tr>
<tr>
<td>/25</td>
<td>255.255.255.128</td>
<td>1/2 C</td>
<td>126</td>
<td>2^(32-25)-2</td>
</tr>
<tr>
<td>/26</td>
<td>255.255.255.192</td>
<td>1/4 C</td>
<td>62</td>
<td>2^(32-26)-2</td>
</tr>
<tr>
<td>/27</td>
<td>255.255.255.224</td>
<td>1/8 C</td>
<td>30</td>
<td>2^(32-27)-2</td>
</tr>
<tr>
<td>/28</td>
<td>255.255.255.240</td>
<td>1/16 C</td>
<td>14</td>
<td>2^(32-28)-2</td>
</tr>
<tr>
<td>/29</td>
<td>255.255.255.248</td>
<td>1/32 C</td>
<td>6</td>
<td>2^(32-29)-2</td>
</tr>
<tr>
<td>/30</td>
<td>255.255.255.252</td>
<td>1/64 C</td>
<td>2</td>
<td>2^(32-30)-2</td>
</tr>
<tr>
<td>/31</td>
<td>255.255.255.254</td>
<td>1/128 C</td>
<td>0</td>
<td>2^(32-31)-2</td>
</tr>
<tr>
<td>/32</td>
<td>255.255.255.255</td>
<td>1/256 C</td>
<td>1</td>
<td>reserved</td>
</tr>
</tbody>
</table>
<hr />
<p>There are <a href="https://en.wikipedia.org/wiki/Reserved_IP_addresses#IPv4">some range of IPv4 address reserved</a> for special uses. More updates coming…</p>
A Walk in the Clouds2022-03-08T00:00:00+00:00https://xjlin0.github.io/tech/2022/03/08/walk-in-the-clouds
<h3 id="push-a-button-and-get-everything-in-the-cloud-">Push a button and get everything in the cloud? <img src="/assets/imgs/cloud.jpeg" alt="Various Edna Mode Styles" width="40%" /></h3>
<p>Recently I joined <a href="https://www.coursera.org/learn/gcp-fundamentals">Coursera’s GCP courses</a> for helping the migration of some services to the GCP cloud. Because of past experience I had some concept of AWS, however GCP world is still pretty new for me. Thus I create a note here for myself to learn <a href="https://explore.skillbuilder.aws/">AWS</a>, <a href="https://docs.microsoft.com/en-us/learn/azure/">Azure</a> and <a href="https://cloud.google.com/training/learning-path-offer">GCP</a>.</p>
<hr />
<table style="width: 100%;">
<thead>
<tr>
<th> </th>
<th><a href="https://aws.amazon.com/">AWS</a></th>
<th><a href="https://azure.microsoft.com/">Azure</a></th>
<th><a href="https://cloud.google.com/">GCP</a></th>
</tr>
</thead>
<tbody>
<tr>
<td>IAAS</td>
<td>—————</td>
<td>————–</td>
<td>————–</td>
</tr>
<tr>
<td>Server</td>
<td>EC2</td>
<td>Azure Virtual Machine</td>
<td>Compute Engine</td>
</tr>
<tr>
<td>OS images</td>
<td>AWS Marketplace</td>
<td>Azure Marketplace</td>
<td>Marketplace</td>
</tr>
<tr>
<td>Object storage</td>
<td>S3</td>
<td>Blob Storage</td>
<td>Cloud Storage</td>
</tr>
<tr>
<td>Relational Database</td>
<td>RDS</td>
<td>Azure Database</td>
<td>Cloud SQL/spanner</td>
</tr>
<tr>
<td>DNS</td>
<td>Route 53</td>
<td>Azure DNS</td>
<td>Cloud DNS</td>
</tr>
<tr>
<td>Serverless</td>
<td>Lambda</td>
<td>Azure Functions</td>
<td>Cloud Functions</td>
</tr>
<tr>
<td>NoSQL database</td>
<td>Document DB, Dynamo DB, Simpler DB</td>
<td>Table Storage, CosmosDB</td>
<td>Cloud Bigtable, DataStore</td>
</tr>
<tr>
<td>Queue</td>
<td>MSK</td>
<td>Kafka on HDInsight</td>
<td>Apache Kafka, Google pub/sub</td>
</tr>
<tr>
<td>permission</td>
<td>IAM <br />(by account)</td>
<td>Azure Active Directory <br /> (by organization)</td>
<td>Cloud IAM <br /> (by project)</td>
</tr>
<tr>
<td>monitoring</td>
<td>Cloudwatch</td>
<td>Azure Monitor</td>
<td>Stackdriver</td>
</tr>
<tr>
<td>PAAS</td>
<td>—————</td>
<td>————–</td>
<td>————–</td>
</tr>
<tr>
<td>platform</td>
<td>Elastic Beanstalk</td>
<td>App Services</td>
<td>App Engine</td>
</tr>
<tr>
<td>Container Orchestration</td>
<td>EKS</td>
<td>AKS</td>
<td>GKE</td>
</tr>
<tr>
<td>Multi Cloud</td>
<td>EKS Anywhere</td>
<td>Azure Migrate</td>
<td>Anthos</td>
</tr>
</tbody>
</table>
<hr />
<p>The major goal of my GCP course study is Kubernetes, K8. The concept of k8 can be simplified by the following drawing:
<img src="/assets/imgs/k8.jpg" alt="objects in Kubernetes" width="100%" /></p>
<p>However, deploying K8 requires some hosts. If you’d like to play your own K8 for free, other than signing up free trial AWS, one way is to <a href="https://andrewlock.net/running-kubernetes-and-the-dashboard-with-docker-desktop/">enable K8 in docker desktop</a> and install Dashboard. Or you can setup <a href="https://k3s.io/">K3s (a lightweight K8 replacement, armv7 binaries available)</a> dashboard on Raspberry Pi. Here are steps based on <a href="https://anthonynsimon.com/blog/kubernetes-cluster-raspberry-pi/">Anthony’s step</a> on Debian 10:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>$ sudo apt upgrade -y
$ sudo apt install -y docker.io
$ sudo docker info
$ sudo sed -i \
'$ s/$/ cgroup_enable=cpuset cgroup_enable=memory cgroup_memory=1 swapaccount=1/' \
/boot/cmdline.txt
$ sudo reboot
$ curl -sfL https://get.k3s.io | sh -
# Create a namespace and and switch to it
$ kubectl create namespace kubernetes-dashboard
$ kubens kubernetes-dashboard
# Deploy from the official source
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.2.0/aio/deploy/recommended.yaml
$ kubectl patch deployment kubernetes-dashboard -n kubernetes-dashboard --type 'json' -p '[{"op": "add", "path": "/spec/template/spec/containers/0/args/-", "value": "--enable-skip-login"}]'
$ kubectl proxy
</code></pre></div></div>
<p>K3s dashboard, <a href="visit http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=_all">http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/#/overview?namespace=_all</a>, on my local Raspberry Pi 3B+ is very slow, but it’s surely fun!</p>
Model Auditing and History in Django2022-02-28T00:00:00+00:00https://xjlin0.github.io/tech/2022/02/28/model-auditing-history-in-django
<h3 id="need-to-track-changes-in-the-past-">Need to track changes in the past? <img src="/assets/imgs/edna_mode.webp" alt="Various Edna Mode Styles" width="40%" /></h3>
<p>There’s a feature request to my meeting management Django App “Attendees”: show all changes history of data, in case something happens. That sounds like a reasonable request since there’s always human error. Sometimes we just need to trace back to find some old data.</p>
<p>There are <a href="https://djangopackages.org/grids/g/model-audit/">many Django packages capable doing model tracking</a>. Here I am listing two interesting packages: <a href="https://github.com/Opus10/django-pghistory">Django-pghistory</a> 1.2.1 and <a href="https://github.com/etianen/django-reversion">Django-reversion</a> 5.0.0. They are using different approaches for tracking the model data changes, so you can find who changed what at when in the past. Both of their documentation explain well on how to start. Again, there’s nothing perfect, please find one that suits you the best.</p>
<hr />
<table style="width: 100%;">
<thead>
<tr>
<th>Item</th>
<th><a href="https://github.com/Opus10/django-pghistory">Django-pghistory</a></th>
<th><a href="https://github.com/etianen/django-reversion">Django-reversion</a></th>
</tr>
</thead>
<tbody>
<tr>
<td>database supported</td>
<td>only Postgres</td>
<td>any database Django supported</td>
</tr>
<tr>
<td>history stored location</td>
<td>Each model has its own db table</td>
<td>All models history in one single db table</td>
</tr>
<tr>
<td>history stored models</td>
<td>Each model has its own history model</td>
<td>All models history share one history model</td>
</tr>
<tr>
<td>History generates by</td>
<td>Postgres triggers automatically</td>
<td><a href="https://django-reversion.readthedocs.io/en/stable/views.html#decorators">Python decorators</a> added by developers</td>
</tr>
<tr>
<td>tracking coverage</td>
<td>everything captured by database</td>
<td>Python code is required to track changes</td>
</tr>
<tr>
<td>grouping multiple models changes</td>
<td><a href="https://django-pghistory.readthedocs.io/en/latest/extras.html#aggregating-events-with-aggregateevent">by AggregateEvent</a></td>
<td><a href="https://django-reversion.readthedocs.io/en/stable/api.html#creating-revisions">by revision block</a></td>
</tr>
<tr>
<td>Rollback/Revert</td>
<td>N/A</td>
<td>supported out of box</td>
</tr>
<tr>
<td>Django admin UI support</td>
<td><a href="https://django-pghistory.readthedocs.io/en/latest/extras.html#showing-event-history-in-the-django-admin">needs coding</a></td>
<td>supported out of box</td>
</tr>
<tr>
<td>Tracking Third-Party Model Changes</td>
<td><a href="https://django-pghistory.readthedocs.io/en/latest/tutorial.html#tracking-third-party-model-changes">set in AppConfig</a></td>
<td><a href="https://django-reversion.readthedocs.io/en/stable/admin.html?#integration-with-3rd-party-apps">re-register by VersionAdmin</a></td>
</tr>
<tr>
<td>Relation tracking by</td>
<td>Context Collection with through models</td>
<td><a href="https://django-reversion.readthedocs.io/en/stable/api.html#registration-api"><code class="language-plaintext highlighter-rouge">register(follow=...)</code></a></td>
</tr>
</tbody>
</table>
<hr />
<p>Django-pghistory tracks model changes by Postgres database triggers, no other databases supported. Once configured a model’s history model “Event”, all changes will be recorded to its history table automatically at database level, without additional codes. So tracking is working out of box after proper configuration. Even changes in Django or PSQL shell will be recored automatically without python code, and the history generation is surely quicker. This is extremely useful if other non-python services also writing to your Django databases. Each model has its own history table, and recorded changes are plain database rows, so querying particular history is straightforward. You can even bundle changes of multiple models to the same context by Event Aggregate. However there’s no active development after May 2021, and the installation/configuration may take some time.</p>
<p>Django-reversion is undoubtedly the most popular and active history tracking package with bigger user community. After registering models in Admin, changes of all registered Django models will be captured in json format at a single database table. History listing and rollback in Admin UI works like a breeze. You can bundle changes of several models by writing revision block codes. However please be prepared to manually add tracking codes to all your Django code changing model data, or tracking won’t work if missed. For example, if there are 4 views changes the model of interest, you will need to add tracking code to all of 4 views. Adding new views/codes without tracking may escape the history recording. Also, querying certain changes in json history from all models may not be very straightforward.</p>
<p>Of course, these are just some of my observations, please let me know if any info are incorrect or changed anytime.</p>
Wordpress on Raspberry Pi with Virtualmin2022-01-30T00:00:00+00:00https://xjlin0.github.io/tech/2022/01/30/wordpress-virtualmin-raspberry
<h3 id="kids-what-is-a-web-server-">Kids: What is a web server? <img src="/assets/imgs/incredibles_computer.jpg" alt="Bob using a computer" width="40%" /></h3>
<p>Last weekend my nephew came over with computer parts and I helped him to assemble his first PC. He’s not sure what to do with his old Raspberry Pi. That’s a great chance to explain the concept of web servers and clients. Compared to k8 & containers, physical things are generally easier to understand for early teenagers.</p>
<p>In the end I setup a tiny Wordpress website on the Raspberry Pi with Virtualmin, a GUI for server administration. Out of my surprise, the Worpress site’s not slow and very smooth for a single visitor, after the site administration done in Virtualmin GUI. Generally the server load is between 0.5~1.0 and consumes no more than 3W. There are <a href="https://smartblogger.com/free-wordpress-hosting/">tons of hosting options</a> but such settings might be useful for demo process later.</p>
<hr />
<table style="width:100%;">
<thead>
<tr>
<th>Item</th>
<th>Version</th>
</tr>
</thead>
<tbody>
<tr>
<td>Server</td>
<td>Raspberry Pi 3B+, with 64G microSD</td>
</tr>
<tr>
<td>OS</td>
<td>Raspberry Pi OS (Debian 10 <strong>Buster</strong> Linux kernel 5.10)</td>
</tr>
<tr>
<td>Virtualmin</td>
<td>6.17 (with Webmin 1.984)</td>
</tr>
</tbody>
</table>
<hr />
<p>1. [OS installation] Because the latest Raspberry Pi OS moved to Debian 11 <strong>Bullseye</strong> after October 2021, please use <a href="https://www.raspberrypi.com/software/">Raspberry Pi Imager</a> to write <a href="https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-legacy">Raspberry Pi OS (Legacy)</a> to the microSD card, since currently Virtualmin installation script only support Debian 10 <strong>Buster</strong>. You may choose to install without Desktop software to make the server cleaner.</p>
<p>2. [Setup swap] Although Virtualmin installation script provide an option to change swap size, I prefer to increase to 1GB swap in Raspberry way:</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>dphys-swapfile swapoff
<span class="nb">sudo </span>nano /etc/dphys-swapfile
<span class="c">#Find the line of CONF_SWAPSIZE=100 and change to 1024 (1GB), save and exit</span>
<span class="nb">sudo </span>dphys-swapfile setup
<span class="nb">sudo </span>dphys-swapfile swapon
</code></pre></div></div>
<p>3. [Change root password] Please DO change default “pi” user’s password since the site maybe public.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>su
passwd
<span class="nb">exit</span>
</code></pre></div></div>
<p>4. [Get a FDQN domain name] This will be needed in the next steps. Since this is an in-home setting, I changed the router’s setting to expose the Raspberry Pi to public by add a Game or NAT mapping on forwarding port 80/443. If you don’t have NoIP.com or DynDNS.org account, nslookup your public ip may get you a temporary FDQN from ISP.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt-get <span class="nt">-y</span> <span class="nb">install </span>dnsutils
nslookup <your public IP>
<span class="c"># find the FDQN domain name that ISP assigned</span>
</code></pre></div></div>
<p>5. [<a href="https://forums.raspberrypi.com/viewtopic.php?t=254307#p1690845">Install Virtualmin on Raspberry Pi</a>] Since Raspberry Pi OS is not officially supported by Virtualmin, we need to force the installation script recognize the system as Debian. The installation may take quite some time.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code>wget http://software.virtualmin.com/gpl/scripts/install.sh
nano install.sh
<span class="c">#Find the text "get_distro", around line # 600</span>
<span class="c">#Add a new line of os_type="debian" immediately after the line of get_distro, save and exit</span>
<span class="nb">sudo </span>sh ./install.sh <span class="c"># it will ask for FDQN domain name</span>
</code></pre></div></div>
<p>6. [Virtualmin post installation] After Virtualmin installation from shell it will ask you to visit Virtualmin page on your Raspberry Pi to finish post-installation settings. Since Virtualmin port is default 10000, you may need to visit something like https://<code class="language-plaintext highlighter-rouge">your Raspberry Pi local ip</code>:10000, and force accept self signed certificate by typing this in developer console. Then login with the root password in step 3.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>sendCommand(SecurityInterstitialCommandId.CMD_PROCEED)
</code></pre></div></div>
<p>Please also add a Virtual server with a new user (owner). And grant the new user more user permission by Virtualmin -> Administration Options -> Edit Owner Limit -> Can install scripts/Can manage databases/Can schedule backups/Can make backups.</p>
<p>7. (optional)[More PHP versions] If newer PHP versions are preferred instead of PHP 7.3 default in Debian 10 Buster, <a href="https://www.itzgeek.com/how-tos/linux/debian/how-to-install-php-7-3-7-2-7-1-on-debian-10-debian-9-debian-8.html">setup repository for PHP now</a>, and <a href="https://www.virtualmin.com/documentation/web/multiplephp/#Installing_PHP_56_andor_74_andor_80_on_Debian_910">you can add multiple PHP versions later</a> but please consider the <a href="https://make.wordpress.org/core/handbook/references/php-compatibility-and-wordpress-versions/">PHP version supported in Wordpress</a> as well as its plugins.</p>
<div class="language-sh highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">sudo </span>apt <span class="nb">install</span> <span class="nt">-y</span> curl wget gnupg2 ca-certificates lsb-release apt-transport-https
wget https://packages.sury.org/php/apt.gpg
<span class="nb">sudo </span>apt-key add apt.gpg
<span class="nb">echo</span> <span class="s2">"deb https://packages.sury.org/php/ </span><span class="si">$(</span>lsb_release <span class="nt">-sc</span><span class="si">)</span><span class="s2"> main"</span> | <span class="nb">sudo tee</span> /etc/apt/sources.list.d/php7.list
<span class="nb">sudo </span>apt update
</code></pre></div></div>
<p>If multiple PHP versions are installed, you may want to assign a default PHP version by visiting System Settings -> Server Templates -> Default -> PHP Options.</p>
<p>8. [Setup Wordpress] Logout Virtualmin page and login with the new user. Visit Virtualmin -> Install Scripts, select Wordpress, install and finish the Wordpress. You may setup automatic SSL renewal for the new site by visiting Virtualmin -> Server Configuration -> SSL Certificate -> Let’s Encrypt. enjoy!</p>
Auto time zone in Django2019-08-30T00:00:00+00:00https://xjlin0.github.io/tech/2019/08/30/auto-time-zone-in-django
<h3 id="time-has-been-changed-">Time has been changed. <img src="/assets/imgs/incredibles_watch_skin.png" alt="watch skin of incredibles" width="20%" /></h3>
<p>The servers running typical web apps are usually in UTC time zone. Servers has no idea of user time zones. When users around the world visiting the website, the time attributes of objects from server can be converted to user local time zones in their browsers. This is a scalable approach, not only such setting can serves users in different time zones, allows spinning up more servers without extra settings, but also fits the design principle for separating concerns.</p>
<p>In Rails there are multiple ways to do such time conversion, <a href="https://medium.com/@theodorelaurent/display-time-in-user-local-time-in-rails-212155fa6447">mostly manually in View layer</a>. How do we setup the time conversion in Django? For display, there is a great Django template filter <code class="language-plaintext highlighter-rouge">|date:'c'</code> can always show ISO 8601 time format for conversion later, but we also want to get user time input correctly converted back to UTC at server.</p>
<p>There is <a href="https://github.com/adamcharnock/django-tz-detect">a great app</a> supposed to do that but unfortunately does not work in my environment. Here is my implementation to achieve it by cookie automatically and system-wisely, without disturbing users.</p>
<p><img src="/assets/imgs/Auto_time_zone_in_Django_by_cookie.png" alt="watch skin of incredibles" width="80%" /></p>
<p>In the JavaScript, upon page load, detecting client time zone and write encoded time zone name (such as ‘America/Los_Angeles’) is possible by calling native <code class="language-plaintext highlighter-rouge">Intl.DateTimeFormat</code> when every page loads:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="kd">const</span> <span class="nx">timeZoneName</span> <span class="o">=</span> <span class="nx">Intl</span><span class="p">.</span><span class="nx">DateTimeFormat</span><span class="p">().</span><span class="nx">resolvedOptions</span><span class="p">().</span><span class="nx">timeZone</span><span class="p">;</span>
<span class="nb">document</span><span class="p">.</span><span class="nx">cookie</span> <span class="o">=</span> <span class="dl">'</span><span class="s1">timezone=</span><span class="dl">'</span> <span class="o">+</span> <span class="nb">encodeURIComponent</span><span class="p">(</span><span class="nx">timeZoneName</span><span class="p">)</span> <span class="o">+</span> <span class="dl">'</span><span class="s1">; path=/</span><span class="dl">'</span><span class="p">;</span></code></pre></figure>
<p>So the cookie now have encoded user time zone data, we just need read cookie, decode and activate the user time zone by marvelous <code class="language-plaintext highlighter-rouge">MiddlewareMixin</code>. For the case of initial visit, we can refer to default setting in settings.py</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/middleware.py
</span><span class="kn">import</span> <span class="nn">pytz</span>
<span class="kn">from</span> <span class="nn">urllib</span> <span class="kn">import</span> <span class="n">parse</span>
<span class="kn">from</span> <span class="nn">django.utils</span> <span class="kn">import</span> <span class="n">timezone</span>
<span class="kn">from</span> <span class="nn">django.utils.deprecation</span> <span class="kn">import</span> <span class="n">MiddlewareMixin</span>
<span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
<span class="k">class</span> <span class="nc">TimezoneMiddleware</span><span class="p">(</span><span class="n">MiddlewareMixin</span><span class="p">):</span>
<span class="o">@</span><span class="nb">staticmethod</span>
<span class="k">def</span> <span class="nf">process_request</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="n">tzname</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">COOKIES</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'timezone'</span><span class="p">)</span> <span class="ow">or</span> <span class="n">settings</span><span class="p">.</span><span class="n">CLIENT_DEFAULT_TIME_ZONE</span>
<span class="n">timezone</span><span class="p">.</span><span class="n">activate</span><span class="p">(</span><span class="n">pytz</span><span class="p">.</span><span class="n">timezone</span><span class="p">(</span><span class="n">parse</span><span class="p">.</span><span class="n">unquote</span><span class="p">(</span><span class="n">tzname</span><span class="p">)))</span>
</code></pre></div></div>
<p>To make user see what time zone is shown, we also declare a time zone variable to store time zone abbreviation, such as <code class="language-plaintext highlighter-rouge">PDT</code>.</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># app/context_processors.py
</span><span class="kn">from</span> <span class="nn">pytz</span> <span class="kn">import</span> <span class="n">timezone</span>
<span class="kn">from</span> <span class="nn">datetime</span> <span class="kn">import</span> <span class="n">datetime</span>
<span class="kn">from</span> <span class="nn">django.conf</span> <span class="kn">import</span> <span class="n">settings</span>
<span class="kn">from</span> <span class="nn">urllib</span> <span class="kn">import</span> <span class="n">parse</span>
<span class="k">def</span> <span class="nf">common_variables</span><span class="p">(</span><span class="n">request</span><span class="p">):</span>
<span class="n">tzname</span> <span class="o">=</span> <span class="n">request</span><span class="p">.</span><span class="n">COOKIES</span><span class="p">.</span><span class="n">get</span><span class="p">(</span><span class="s">'timezone'</span><span class="p">)</span> <span class="ow">or</span> <span class="n">settings</span><span class="p">.</span><span class="n">CLIENT_DEFAULT_TIME_ZONE</span>
<span class="k">return</span> <span class="p">{</span>
<span class="s">"timezone_name"</span><span class="p">:</span> <span class="n">datetime</span><span class="p">.</span><span class="n">now</span><span class="p">(</span><span class="n">timezone</span><span class="p">(</span><span class="n">parse</span><span class="p">.</span><span class="n">unquote</span><span class="p">(</span><span class="n">tzname</span><span class="p">))).</span><span class="n">tzname</span><span class="p">()</span>
<span class="p">}</span>
</code></pre></div></div>
<p>And then add our new magic TimezoneMiddleware and the new variable in the Django settings.py:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">CLIENT_DEFAULT_TIME_ZONE</span> <span class="o">=</span> <span class="s">"pick a time zone, i.e. America/Los_Angeles"</span>
<span class="n">MIDDLEWARE</span> <span class="o">=</span> <span class="p">[</span>
<span class="c1">## other Middlewares
</span> <span class="s">"app.middleware.TimezoneMiddleware"</span><span class="p">,</span>
<span class="p">]</span>
<span class="n">TEMPLATES</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="c1">### other settings
</span> <span class="s">"context_processors"</span><span class="p">:</span> <span class="p">[</span>
<span class="c1">### other context_processors
</span> <span class="s">"app.context_processors.common_variables"</span><span class="p">,</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
</code></pre></div></div>
<p>It is fine to show user what time zone is being used, even <a href="https://en.wikipedia.org/wiki/List_of_time_zone_abbreviations">the abbreviations are not unique</a>, for example, AST stands for Arabia Standard Time (UTC+3) or Atlantic Standard Time (UTC-4).</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="nt"><p></span>
You last visited at {{ current_user.last_visited_at }} {{timezone_name}}
<span class="nt"></p></span>
When do you want to go?
<span class="nt"><input</span> <span class="na">type=</span><span class="s">"datetime-local"</span> <span class="na">name=</span><span class="s">"departure-time"</span><span class="nt">></span>
</code></pre></div></div>
<p>And voilà! the time will be shown in user time zone, even it was originally stored on the server in UTC time zone. Moreover, when user inputs their time, the saved record at the server will be automatically converted back to UTC. Isn’t that great?</p>
<p>The caveat of the implementation can be spotted when user system time changes due to daylight saving time or user traveling. If user still have the session, he/she will see the page remain in the old time timezone. It may not be a common issue with shorter SESSION_COOKIE_AGE. Since there’s no browser event for system time change, the workaround is to reload page or periodically update cookie by JavaScript.</p>
Running native queries in ORM2018-10-30T00:00:00+00:00https://xjlin0.github.io/tech/2018/10/30/running-native-queries-in-orm
<h3 id="sometimes-the-orm-mathods-is-just-not-enough-">Sometimes the ORM mathods is just not enough. <img src="/assets/imgs/db_queries.jpg" alt="finding out who knows the top secret" width="20%" /></h3>
<p>Object-Relational Mapping (ORM) is great! There are some many methods immediately available without crafting SQL, we can often even get the results without crazy join. Filters, aggregations…, etc. you name it!</p>
<p>However, no tools are 100% perfect, sometimes, we just have to join 10 tables to get the desired results. But how to run native SQL queries in ORM? The followings are two ways I like in Rails. Sometimes the results of the raw SQL needed to be map to existing classes. Sometimes it is enough to get array of hashes. It is elegant to use <code class="language-plaintext highlighter-rouge">ActiveRecord::Result</code>’s <code class="language-plaintext highlighter-rouge">columns</code> when the attributes are needed.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># Map results to instances of an existing class</span>
<span class="n">supermen</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">find_by_sql</span><span class="p">(</span><span class="s1">'SELECT * FROM users WHERE super_power = 1'</span><span class="p">)</span>
<span class="c1"># Map results to ActiveRecord::Result</span>
<span class="n">users</span> <span class="o">=</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection_pool</span><span class="p">.</span><span class="nf">with_connection</span> <span class="k">do</span> <span class="o">|</span><span class="n">connection</span><span class="o">|</span>
<span class="n">connection</span><span class="p">.</span><span class="nf">exec_query</span><span class="p">(</span><span class="s1">'SELECT * FROM users'</span><span class="p">)</span> <span class="c1"># Map results to ActiveRecord::Result</span>
<span class="c1">#connection.select_all('SELECT * FROM users') # Map results to Array of Hashes</span>
<span class="k">end</span>
<span class="n">user_count</span> <span class="o">=</span> <span class="n">users</span><span class="p">.</span><span class="nf">count</span>
<span class="n">last_user</span> <span class="o">=</span> <span class="no">User</span><span class="p">.</span><span class="nf">instantiate</span><span class="p">(</span><span class="n">users</span><span class="p">.</span><span class="nf">last</span><span class="p">)</span>
</code></pre></div></div>
<p>But what if you are using JPA? If no proper named queries can be found, can we also run native queries? You bet!</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">package</span> <span class="nn">com.supermen.incredibles.repository</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">com.supermen.incredibles.model.UserEntity</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.springframework.data.jpa.repository.JpaRepository</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.springframework.data.jpa.repository.Query</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">org.springframework.data.repository.query.Param</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">javax.inject.Named</span><span class="o">;</span>
<span class="kn">import</span> <span class="nn">java.util.List</span><span class="o">;</span>
<span class="nd">@Named</span>
<span class="kd">public</span> <span class="kd">interface</span> <span class="nc">UserRepository</span> <span class="kd">extends</span> <span class="nc">JpaRepository</span><span class="o"><</span><span class="nc">UserEntity</span><span class="o">,</span> <span class="nc">String</span><span class="o">></span> <span class="o">{</span>
<span class="nd">@Query</span><span class="o">(</span> <span class="c1">// and join other 10 tables if needed</span>
<span class="n">value</span> <span class="o">=</span> <span class="s">"SELECT * FROM users WHERE name = :name"</span><span class="o">,</span>
<span class="n">nativeQuery</span> <span class="o">=</span> <span class="kc">true</span>
<span class="o">)</span>
<span class="nc">List</span><span class="o"><</span><span class="nc">UserEntity</span><span class="o">></span> <span class="nf">findUsersByRawSql</span><span class="o">(</span><span class="nd">@Param</span><span class="o">(</span><span class="s">"name"</span><span class="o">)</span> <span class="nc">String</span> <span class="n">name</span><span class="o">);</span>
<span class="o">}</span>
</code></pre></div></div>
<p>OK, if using Python/Django, could I also run native queries? Here is how:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># map the results to queryset of Model
</span><span class="n">user</span> <span class="o">=</span> <span class="n">User</span><span class="p">.</span><span class="n">objects</span><span class="p">.</span><span class="n">raw</span><span class="p">(</span><span class="s">'SELECT * FROM users WHERE name = %s'</span><span class="p">,</span> <span class="p">[</span><span class="s">"jack"</span><span class="p">])</span>
<span class="c1"># please pass parameters as a list to avoid SQL injection attacks.
</span>
<span class="c1"># direct running queries
</span><span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">connection</span>
<span class="kn">from</span> <span class="nn">collections</span> <span class="kn">import</span> <span class="n">namedtuple</span>
<span class="c1"># use it like a list of tuples like some_users[0]
</span><span class="k">with</span> <span class="n">connection</span><span class="p">.</span><span class="n">cursor</span><span class="p">()</span> <span class="k">as</span> <span class="n">cursor</span><span class="p">:</span>
<span class="n">cursor</span><span class="p">.</span><span class="n">execute</span><span class="p">(</span><span class="s">"SELECT * FROM users WHERE name = %s"</span><span class="p">,</span> <span class="p">[</span><span class="s">"Bob"</span><span class="p">])</span>
<span class="n">some_users</span> <span class="o">=</span> <span class="n">cursor</span><span class="p">.</span><span class="n">fetchall</span><span class="p">()</span>
<span class="c1"># use it like a list of objects like other_users[0].name
</span><span class="k">with</span> <span class="n">connection</span><span class="p">.</span><span class="n">cursor</span><span class="p">()</span> <span class="k">as</span> <span class="n">cursor</span><span class="p">:</span>
<span class="n">cursor</span><span class="p">.</span><span class="n">execute</span><span class="p">(</span><span class="s">"SELECT * FROM users WHERE name = %s"</span><span class="p">,</span> <span class="p">[</span><span class="s">"Parr"</span><span class="p">])</span>
<span class="n">nt_result</span> <span class="o">=</span> <span class="n">namedtuple</span><span class="p">(</span><span class="s">'Result'</span><span class="p">,</span> <span class="p">[</span><span class="n">col</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="k">for</span> <span class="n">col</span> <span class="ow">in</span> <span class="n">cursor</span><span class="p">.</span><span class="n">description</span><span class="p">])</span>
<span class="n">other_users</span> <span class="o">=</span> <span class="p">[</span><span class="n">nt_result</span><span class="p">(</span><span class="o">*</span><span class="n">row</span><span class="p">)</span> <span class="k">for</span> <span class="n">row</span> <span class="ow">in</span> <span class="n">cursor</span><span class="p">.</span><span class="n">fetchall</span><span class="p">()]</span>
</code></pre></div></div>
<p>That’s it. Enjoy!</p>
Suggesting friends by MapReduce2015-08-30T00:00:00+00:00https://xjlin0.github.io/tech/2015/08/30/suggesting-friends-by-mapreduce
<h3 id="implemented-in-apache-spark-or-hadoop-mrjob-">Implemented in Apache Spark or Hadoop MrJob. <img src="/assets/imgs/avengers.jpg" alt="The Incredibles friends from Pixar" width="20%" /></h3>
<p>One interesting example to practice is the classical question: “The people you may know”. On apps similar to Facebook, you got users and their friends list as the followed. How to suggest user A and E to make friend to each other but not to User K? (based on User A and E get several common friends B, C, & D, and we assume only mutual friendship existed.)</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A B, C, D, G #User A has user B/C/D/G as friends
E B, C, D
K L, J, G
.....
</code></pre></div></div>
<p>Let’s take a look at the input again, since we assume only mutual friendship existed, the real input would extend to the following, how would you solve it?</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A B, C, D, G
B A, E #Because user B is user A and user E's friend.
C A, E
D A, E
E B, C, D
G A, K
J K
K L, J, G
L K
</code></pre></div></div>
<p>The most straightforward approach is to iterate all users. For each user, compare his/her friends to <strong>all other users</strong> for suggestion. This approach is feasible if applied asynchronously because people adding friends at different time. However, if there’s ton’s of users/friends and you wanna to calculate all suggestions at once, a more scalable approach is required.</p>
<p><a href="https://en.wikipedia.org/wiki/MapReduce">MapReduce</a> is one of Google’s approaches for processing big data, and currently there are many implementations based on the idea, such as <a href="https://en.wikipedia.org/wiki/Apache_Hadoop">Apache Hadoop</a> or <a href="https://en.wikipedia.org/wiki/Apache_Spark">Spark</a>, etc. If we can find a way to count common friends, independently, we can split such big job to many workers and make it parallel.</p>
<p>What is the “independently” mean? During the processing of each line in the input, the straightforward approach needs to compare a line to <strong>another</strong> line. That is not independent. There is actually a way to count common friends of one line, <strong>without</strong> reading all other lines. In such way huge inputs can be split to different workers for parallel processing without inter communication in the beginning of the common friend counting. How is it possible? It’s based on one fact: A certain user’s friends all have one common friend, which is that certain user.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>A B, C, D, G
B & C have one common friend (A)
B & D have one common friend (A)
B & G have one common friend (A)
C & D have one common friend (A)
.....so on so forth....
</code></pre></div></div>
<p>Thus when reading line for user A, I first make all permutation pair within that line. There are two kinds of pair here:</p>
<ul>
<li>
<p>[<em>connected</em>] Those who are already connected friends, and I assign their ‘common friends count’ as a very negative number -9999999999 as my personal label for whoever are already friends. Thus I won’t suggest friendship request for users who are already friends, since 9999999999 is bigger than the world population.</p>
</li>
<li>
<p>[<em>commons</em>] Those who have user A as the common friend, and I assigned these pair’s “common friends count” as one, since they have <strong>one</strong> common friend user A. Here is the example:</p>
</li>
</ul>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1"># input (first line): A: B, C, D, G
</span><span class="n">connected</span> <span class="o">=</span> <span class="p">[((</span><span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="o">-</span><span class="mi">9999999999</span><span class="p">),</span> <span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="o">-</span><span class="mi">9999999999</span><span class="p">),</span> <span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">D</span><span class="p">),</span> <span class="o">-</span><span class="mi">9999999999</span><span class="p">),</span> <span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">G</span><span class="p">),</span> <span class="o">-</span><span class="mi">9999999999</span><span class="p">)]</span>
<span class="n">commons</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">((</span><span class="n">B</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">C</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">B</span><span class="p">,</span> <span class="n">D</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">D</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">B</span><span class="p">,</span> <span class="n">G</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">G</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span>
<span class="p">((</span><span class="n">C</span><span class="p">,</span> <span class="n">D</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">D</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">C</span><span class="p">,</span> <span class="n">G</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">G</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span>
<span class="p">((</span><span class="n">D</span><span class="p">,</span> <span class="n">G</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="p">((</span><span class="n">G</span><span class="p">,</span> <span class="n">D</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span>
<span class="p">]</span>
<span class="k">return</span> <span class="n">connected</span> <span class="o">+</span> <span class="n">commons</span>
<span class="c1"># repeat for all lines</span></code></pre></figure>
<p>You may think, wow, why generate or <strong>map</strong> to so many ((pair), count) from one single line? Is it necessary? The point is such mapping process is independently line to line, getting their counts (-9999999999 or 1) doesn’t depend on reading any other lines. Thus the mapping process of huge input can split into many workers for parallel processing.</p>
<p>The next step is to use the pair as the key, sum up or <strong>reduce</strong> their counts. So after reduce, the sum of these pair from all three lines would like the following: Totally there are two pairs of (A, B), and the sum of the two pairs would be -19999999998. There would be 3 pairs of (A, E), and the sum of the three pairs would be 3. Please note here the summing/reduce process can be executed on different workers, the final reduced result won’t be changed.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1"># Some sums of the pairs (reduced results)
</span><span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="o">-</span><span class="mi">19999999998</span><span class="p">),</span> <span class="c1">#the pair are generated from user A and user B, the counts are the sum.
</span><span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="o">-</span><span class="mi">19999999998</span><span class="p">),</span>
<span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">D</span><span class="p">),</span> <span class="o">-</span><span class="mi">19999999998</span><span class="p">),</span>
<span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">G</span><span class="p">),</span> <span class="o">-</span><span class="mi">19999999998</span><span class="p">),</span>
<span class="p">((</span><span class="n">E</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="o">-</span><span class="mi">19999999998</span><span class="p">),</span>
<span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">E</span><span class="p">),</span> <span class="mi">3</span><span class="p">),</span> <span class="c1">#The pair user A and user E have counts 3 from the lines of user B/C/D
</span><span class="p">((</span><span class="n">E</span><span class="p">,</span> <span class="n">A</span><span class="p">),</span> <span class="mi">3</span><span class="p">),</span> <span class="c1">#permutation, not combination. so you can also suggest user E to make friend with user A later.
</span><span class="p">((</span><span class="n">B</span><span class="p">,</span> <span class="n">C</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span> <span class="c1">#they have 2 common friends: A, E
</span><span class="p">((</span><span class="n">C</span><span class="p">,</span> <span class="n">B</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span>
<span class="p">((</span><span class="n">A</span><span class="p">,</span> <span class="n">K</span><span class="p">),</span> <span class="mi">1</span><span class="p">),</span> <span class="c1">#Generated from the line of user G
#....</span></code></pre></figure>
<p>Now we see the pattern: The reduced results, which is the counts, are the number of common friends. Don’t forget to filter the positive counts and the pair will be the suggestions with the common friend count. We just need to group the same leading users and suggest the most suggested pairs based on their counts.</p>
<p>Based on the same algorithm described above, here are my two implementations, by using two different frameworks. The first one is Python API to the cluster computing framework <a href="http://spark.apache.org">Apache Spark</a>. It comes with many handy functions such as groupByKey() or takeOrdered() for grouping pair-counts or choosing the top pair-counts. You can check <a href="https://github.com/xjlin0/cs246/blob/master/w2015/hw1/q1_people_you_may_know_spark.py">my PySpark code here</a>. In addition, by using <a href="http://ondra-m.github.io/ruby-spark/"><strong>Ruby-Spark</strong> gem</a>, I also coded a <a href="https://github.com/xjlin0/cs246/blob/master/w2015/hw1/q1_people_you_may_know_spark.rb">Ruby-Spark solution</a> for you to compare.</p>
<p>My second implementation used <a href="https://github.com/Yelp/mrjob">MRJob</a>, the Yelp’s Python API controlling Hadoop for MapReduce tasks. There are not too many built-in functions so I have hard time to implement the groupByKey() or top counts. After struggling here is my working code of groupByKey() in Yelp MrJob:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c1"># Input: ((pair), [count] )
# Please notify the count are wrapped in the []
</span>
<span class="k">def</span> <span class="nf">groupByKey</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">key</span><span class="p">,</span> <span class="n">values</span><span class="p">):</span>
<span class="k">return</span> <span class="n">key</span><span class="p">,</span> <span class="nb">reduce</span><span class="p">(</span><span class="k">lambda</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">:</span> <span class="n">a</span> <span class="o">+</span> <span class="n">b</span><span class="p">,</span> <span class="n">values</span><span class="p">)</span>
<span class="c1"># [ ] + [ ] is possible, so ((pair), [a] ) and ((pair), [b] ) will be ((pair), [a, b] )</span></code></pre></figure>
<p>I used the <a href="https://docs.python.org/2/library/collections.html">Python’s Counter object</a> to get top counts by its most_common() function. Welcome to check my <a href="https://github.com/xjlin0/cs246/blob/master/w2015/hw1/q1_people_you_may_know_mrjob.py">MRjob implementation</a>. Enjoy!</p>
Great female engineers2015-07-31T00:00:00+00:00https://xjlin0.github.io/cultural/2015/07/31/great-female-engineers
<h3 id="are-you-the-victim-of-sexual-stereotyping">Are you the victim of sexual stereotyping?<img src="/assets/imgs/helenkick.jpg" alt="Helen Kicking" width="20%" /></h3>
<p>Varieties stimulate thinkings from more angles, and different thinkings trigger creativities. Historically male is dominant in the engineer field due to many reasons. But sometimes people misread such trend as “female is not a good fit in certain fields”. I would like to argue the opposite by discussing some examples.</p>
<p><a href="https://en.wikipedia.org/wiki/Grace_Hopper">Admiral Grace Hopper, the “Amazing Grace”</a> earned her degree in mathematics and started her faculty life in 1930s. During the WWII she expressed the extreme courage by voluntarily joining Navy. She <a href="https://en.wikipedia.org/wiki/Debugging#Origin">debug</a> as one of the first full-time developers on <a href="https://en.wikipedia.org/wiki/Harvard_Mark_I#Contribution_to_the_Manhattan_project">Harvard Mark computers</a>, a military development later used in the Manhattan project. In her career, she created the first modern compiler <a href="https://en.wikipedia.org/wiki/A-0_System">A-0</a> on <a href="https://en.wikipedia.org/wiki/UNIVAC_I">UNIVAC</a>, created the foundation of the calculation specialized, data orientated higher language <a href="https://en.wikipedia.org/wiki/COBOL">COBOL</a>, implemented numerous app standard tests later adopted by <a href="https://en.wikipedia.org/wiki/National_Institute_of_Standards_and_Technology">NIST</a>. She retired three times from Navy but always called back to deal with problems that nobody else can solve. <a href="https://www.nersc.gov/users/computational-systems/hopper/">Cray XE6 supercomputer at NERSC</a> was recently named after her, ironically, in 70s she foresaw and persuaded the military to use clustered small computers on network instead of using mainframe centralized supercomputers. What a cloud platform prophet!</p>
<p><a href="https://en.wikipedia.org/wiki/Margaret_Hamilton_(scientist)">Margaret Hamilton</a>, the outstanding computer scientist, also earned her degree in mathematics, published over 130 papers and successfully participated in numerous projects. In 1960s She was one of the few developers for <a href="https://en.wikipedia.org/wiki/AN/FSQ-7_Combat_Direction_Central">the RADAR computer XD-1</a>. She was so advanced in the art of the software development with extraordinary concepts of system design/development, process modeling and code reusabilities. She was joked and mocked when first using the term of “software engineering” to describe such sophisticated process of art. As the lead software engineer in NASA, her navigation apps was strong enough to sustained, recovered and eventually saved the moon landing of <a href="https://en.wikipedia.org/wiki/Apollo_program">Apollo 11</a>. Triggered <a href="https://www.hq.nasa.gov/alsj/a11/a11.1201-pa.html">alarm 1201/1202</a>, this critical app didn’t brake even in the unhappy path due to extra loading by error inputs. Yes, it’s the fact, <code>an exceptional woman saved a manned lunar program</code>.</p>
<p>There are many more great female engineers such as the first programmer <a href="https://en.wikipedia.org/wiki/Ada_Lovelace#First_computer_program">Ada Lovelace</a> and the leader <a href="https://en.wikipedia.org/wiki/Beatrice_Hicks">Beatrice Hicks</a>. But I’d like to switch to my real experience in my daily life. my mom taught me elementary arithmetic on the blackboard, and both of my excellent math teachers in primary and middle school were chosen to lead the special math teaching class. (For the record, teaching naughty kids logic concept is not any easier than lunar project by the way). Today I am still benefit by the foundation learned from them. In Dev Bootcamp <a href="https://twitter.com/mashalifshin">Masha</a> showed me the wonder of ORM and <a href="https://twitter.com/jay_gee">Jen</a> taught me the beauty of the Rails. I wouldn’t survive interview white board without <a href="https://twitter.com/downsamelia">Amelia</a>’s teaching of algorithm. Many of my female cohort mates are marvelous too: <a href="https://twitter.com/CariWest">Cari</a> showed me the variables reference in Ruby, and <a href="https://twitter.com/gz_yumiko">Yumiko</a> showed me the magic in JavaScript. Too many female achievements to count here really.</p>
<p>Seriously, are you still think engineer positions are only for male?</p>
Install Apache Spark with Python or Ruby-Spark in Yosemite2015-06-19T00:00:00+00:00https://xjlin0.github.io/tech/2015/06/19/install-apache-spark-with-python-or-ruby-spark-in-yosemite
<h3 id="enjoy-the-power-of-the-spark-in-local-repl-shell-including-ruby-">Enjoy the power of the spark in local REPL shell, including Ruby. <img src="/assets/imgs/spark.jpg" alt="Syndrome making spark" width="20%" /></h3>
<p>Recently joined the <a href="https://www.edx.org/course/introduction-big-data-apache-spark-uc-berkeleyx-cs100-1x">“EdX’s BerkeleyX CS 100.1X: Introduction to Big Data with Apache Spark”</a> , I am excited to know there’s so much we can do with this cluster computing framework. Here is how I followed <a href="http://sourabhbajaj.com/mac-setup/Vagrant/README.html">Sourabh Bajaj’s tutorial</a> to setup Virtual Box and Vagrant, to extract and run <a href="https://github.com/spark-mooc/mooc-setup/archive/master.zip">the course Vagrantfile</a>. Then a command like <code>vagrant up --provider=virtualbox</code> will get you the complete environment accessible from <a href="http://ipython.org/notebook.html">IPython Notebook</a> within browsers.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><span class="go">brew tap caskroom/cask
brew install brew-cask
brew update && brew upgrade brew-cask && brew cleanup
brew cask install virtualbox
brew cask install vagrant</span></code></pre></figure>
<p>It’s great to have virtual box running. But I really like to have a separate How local REPL environment to play. Now it’s possible to get latest Spark by <code>brew install scala sbt hadoop apache-spark</code>, <code>dev/change-version-to-2.11.sh</code> then <code>build/sbt -Pyarn -Phadoop-2.6 -Dscala-2.11 assembly</code>, with great help such as <a href="http://amodernstory.com/2015/03/05/installing-and-running-spark-with-python-notebook-on-mac/">Marek’s tutorial</a> and <a href="http://blog.prabeeshk.com/blog/2015/04/07/self-contained-pyspark-application/">Prabeesh’s tutorial</a> (<a href="http://ramhiser.com/2015/02/01/configuring-ipython-notebook-support-for-pyspark/">John’s tutorial</a> didn’t worked for me). But <string>Spark 1.3.1 binaries work out of box</string>, deliver its environment painlessly within minutes if you just want to write Spark code immediately, without worrying settings for Hadoop, <a href="http://www.jenv.be/">jEnv</a> or <a href="https://github.com/lieldulev/brew-scala-switcher">scala switcher</a>.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><span class="go">brew install wget python
wget -c http://mirrors.ibiblio.org/apache/spark/spark-1.3.1/spark-1.3.1-bin-hadoop2.6.tgz
tar xvzf ./spark-1.3.1-bin-hadoop2.6.tgz
cd spark-1.3.1-bin-hadoop2.6
bin/pyspark</span></code></pre></figure>
<p>Now your local Python Spark REPL is ready! You can test some codes like this (Clark Updike’s example):</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">Welcome</span> <span class="n">to</span>
<span class="n">____</span> <span class="n">__</span>
<span class="o">/</span> <span class="n">__</span><span class="o">/</span><span class="n">__</span> <span class="n">___</span> <span class="n">_____</span><span class="o">/</span> <span class="o">/</span><span class="n">__</span>
<span class="n">_</span>\ \<span class="o">/</span> <span class="n">_</span> \<span class="o">/</span> <span class="n">_</span> <span class="err">`</span><span class="o">/</span> <span class="n">__</span><span class="o">/</span> <span class="s">'_/
/__ / .__/\_,_/_/ /_/\_\ version 1.3.1
/_/
>>> sc
<pyspark.context.SparkContext object at 0x10ba5fd10>
>>> rdd=sc.parallelize( [('</span><span class="n">a</span> <span class="sa">b</span><span class="s">',(1,'</span><span class="n">gold</span><span class="s">')), ('</span><span class="n">a</span> <span class="sa">b</span><span class="s">',(2,'</span><span class="n">gold</span><span class="s">')), ('</span><span class="n">a</span> <span class="n">c</span><span class="s">',(4,'</span><span class="n">gold</span><span class="s">'))] )
>>> rdd.cache()
#ParallelCollectionRDD[1] at parallelize at PythonRDD.scala:392
>>> rdd.aggregate(0, lambda acc, (k, (v, label)): acc + v, lambda a, b : a + b )
# => 7 # Spark RDD works!</span></code></pre></figure>
<p>Moreover, now it’s possible to use Apache Spark in Ruby by <a href="http://ondra-m.github.io/ruby-spark/">RubySpark gem</a>, in addition to Java, Scala, Python or R. This fabulous gem has not supported DataFrames yet, and its shell is based on pry gem. One of my glitches was pry’s incompatibility with pry-debugger in Ruby 2.2.2, so I had to <code>gem uninstall pry-debugger</code> and made sure pry works alone first.</p>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><span class="go">brew install java sbt
</span><span class="gp">echo 'export JAVA_HOME=$</span><span class="o">(</span>/usr/libexec/java_home<span class="o">)</span><span class="s1">' >> ~/.bash_profile
</span><span class="go">gem install pry ruby-spark
ruby-spark build
ruby-spark shell</span></code></pre></figure>
<p>After compiling for some time, your access to Apache Spark™ is now granted!. Let’s write some RDD now. According to Ruby-spark gem author’s benchmark, starting it by <code>SPARK\_RUBY\_SERIALIZER="oj" ruby-spark shell</code> would use oj as the serializer and will be faster than default Marshal serializer. And the <a href="https://github.com/ondra-m/ruby-spark/wiki/Passing-function">passing of the anonymous function is different from native Ruby</a>. However I don’t care, it just works! have a lot of fun!</p>
<figure class="highlight"><pre><code class="language-irb" data-lang="irb"><span class="go"> Welcome to
__ ____ __
______ __/ / __ __ / __/__ ___ _____/ /__
/ __/ // / _ \/ // / _\ \/ _ \/ _ `/ __/ '_/
/_/ \_,_/_.__/\_, / /___/ .__/\_,_/_/ /_/\_\ version 1.2.0
/___/ /_/
</span><span class="err">
</span><span class="go"> Spark context is loaded as $sc
</span><span class="gp">[1] pry(main)></span><span class="w"> </span><span class="n">rdd</span> <span class="o">=</span> <span class="vg">$sc</span><span class="p">.</span><span class="nf">parallelize</span><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">10</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="p">=></span> <span class="kt">#<</span><span class="no">Spark</span><span class="o">::</span><span class="no">RDD</span><span class="p">:</span><span class="mh">0x70362396061240</span>
<span class="no">Serializer</span><span class="p">:</span> <span class="s2">"Batched(1024) -> Marshal"</span>
<span class="no">Deserializer</span><span class="p">:</span> <span class="s2">"Batched(1024) -> Marshal"</span><span class="kt">></span>
<span class="gp">[2] pry(main)></span><span class="w"> </span><span class="n">rdd</span><span class="p">.</span><span class="nf">cache</span>
<span class="c">#=> #<Spark::RDD:0x70362396061240 (cached) ....
</span><span class="gp">[3] pry(main)></span><span class="w"> </span><span class="n">rdd</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span> <span class="nb">lambda</span><span class="p">{</span><span class="o">|</span><span class="n">x</span><span class="o">|</span> <span class="n">x</span><span class="o">+</span><span class="mi">5</span><span class="p">}</span> <span class="p">).</span><span class="nf">collect</span><span class="p">()</span>
<span class="c">#=> [6, 7, 8, 9, 10, 11, 12, 13, 14, 15] # Spark RDD works!</span></code></pre></figure>
Faster Parsing CSV With Parallel Processing2015-05-25T00:00:00+00:00https://xjlin0.github.io/tech/2015/05/25/faster-parsing-csv-with-parallel-processing
<h3 id="speed-up-parsing-csv-with-smarter_csv-and-parallel-gem">Speed up parsing csv with smarter_csv and parallel gem<img src="/assets/imgs/dash5.jpg" alt="five Dashes" width="20%" /></h3>
<p>One hard moment of my last hackathon project was to read <a href="http://ruby-doc.org/stdlib-2.2.3/libdoc/csv/rdoc/CSV.html">CSV</a> files containing millions of records and store in database, and it took very long time. Are there a faster way?</p>
<p>Any version after Ruby 1.9 or Rails 2.2 is ready for <a href="http://ruby-doc.org/core-2.2.3/Thread.html">Thread</a>, which is a light-weight process. What is Thread for? If a job needs a person working for 10 hours to finish, ideally this job can be finished by 10 person working for 1 hour. A thread can be regard as the individual running environment for a worker. We can assign jobs for many workers in different threads (or processes) to do things simultaneously. Therefore, time consuming processing could be speed up by proper coding, because all jobs are executed simultaneously instead of waiting one by one.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="k">def</span> <span class="nf">talk</span><span class="p">(</span><span class="n">person</span><span class="p">)</span>
<span class="nb">puts</span> <span class="s2">"Start to talk with </span><span class="si">#{</span><span class="n">person</span><span class="si">}</span><span class="s2">"</span>
<span class="nb">sleep</span> <span class="nb">rand</span> <span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">10</span><span class="p">)</span>
<span class="nb">puts</span> <span class="s2">"done talking with </span><span class="si">#{</span><span class="n">person</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="n">persons</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"Bob"</span><span class="p">,</span> <span class="s2">"Helen"</span><span class="p">,</span> <span class="s2">"Violet"</span><span class="p">,</span> <span class="s2">"Dash"</span><span class="p">,</span> <span class="s2">"Jack"</span><span class="p">]</span>
<span class="n">threads</span> <span class="o">=</span> <span class="n">persons</span><span class="p">.</span><span class="nf">map</span> <span class="k">do</span> <span class="o">|</span><span class="n">person</span><span class="o">|</span>
<span class="no">Thread</span><span class="p">.</span><span class="nf">new</span> <span class="p">{</span> <span class="n">talk</span><span class="p">(</span><span class="n">person</span><span class="p">)</span> <span class="p">}</span>
<span class="k">end</span>
<span class="n">threads</span><span class="p">.</span><span class="nf">each</span><span class="p">{</span><span class="o">|</span><span class="n">t</span><span class="o">|</span><span class="n">t</span><span class="p">.</span><span class="nf">join</span><span class="p">}</span>
<span class="nb">puts</span> <span class="s2">"Finished talking with Incredibles!"</span></code></pre></figure>
<p>However, these threads might step in each other’s foot, if accessing shared resources. Such scenario, named <a href="http://en.wikipedia.org/wiki/Race_condition">Race Conditions</a>, need to be considered. For example, if we setup 10 workers to read or write the same file, these workers wouldn’t know other workers’ progress and might process the same line or write the same files to waste time or corrupt data. You can use the <a href="http://ruby-doc.org/core-2.2.3/Mutex.html">Mutex</a>, a locking mechanism, to avoid this, or to split jobs to ensure each workers doing different jobs.</p>
<p>How to split huge CSV files for many workers to parse? There is a amazing gem called <a href="http://github.com/tilo/smarter_csv">smarter_csv</a>, which can parse and split CSV to array of hashes by CSV headers. Its splitting is ideal for parallel processing, such as <a href="http://github.com/resque/resque">Resque</a> or <a href="http://github.com/mperham/sidekiq">Sidekiq</a>. Here is how I split CSV and assign for simultaneously parsing huge CSV files. It shortened days of work to hours by using all 8 Threads on my Mac!</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">require</span> <span class="s1">'parallel'</span>
<span class="nb">require</span> <span class="s1">'smarter_csv'</span>
<span class="k">def</span> <span class="nf">worker</span><span class="p">(</span><span class="n">array_of_hashes</span><span class="p">)</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">reconnect!</span>
<span class="c1">#data seeding</span>
<span class="k">end</span>
<span class="n">chunks</span> <span class="o">=</span> <span class="no">SmarterCSV</span><span class="p">.</span><span class="nf">process</span><span class="p">(</span><span class="s1">'filename.CSV'</span><span class="p">,</span> <span class="ss">chunk_size: </span><span class="mi">1000</span><span class="p">)</span>
<span class="no">Parallel</span><span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="n">chunks</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">chunk</span><span class="o">|</span>
<span class="n">worker</span><span class="p">(</span><span class="n">chunk</span><span class="p">)</span>
<span class="k">end</span></code></pre></figure>
<p>In the above example, the records in the huge csv file are parsed as hashes, with the CSV header as keys. Smarter_csv actually group these hashes into arrays, 1000 hashes/records per array in this example. I then pass the array (chunk) using another fabulous gem <a href="http://github.com/grosser/parallel">parallel</a>. In such way a huge CSV is sliced into chunks of 1000 records and assign to workers for parallel processing, without queuing by <a href="http://redis.io">Redis</a>. One trick mentioned by <a href="http://ruby.zigzo.com/2012/01/29/the-parallel-gem-and-postgresql-oh-and-rails/">Mario</a>, is to reconnect the database since PostgreSQL does not allow using the same connection for more than one thread.</p>
<p>Also, if you just need to update some attributes for the model, you can also utilize all processors cores by doing the following steps. In ActiveRecord, I tend to use <code class="language-plaintext highlighter-rouge">find_each</code> or <code class="language-plaintext highlighter-rouge">find_in_batches</code> to process many records to avoid choking my server.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="nb">require</span> <span class="s1">'parallel'</span>
<span class="k">def</span> <span class="nf">worker</span><span class="p">(</span><span class="n">array_of_instances</span><span class="p">)</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">reconnect!</span>
<span class="n">array_of_instances</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">instance</span><span class="o">|</span>
<span class="n">instance</span><span class="p">.</span><span class="nf">update</span><span class="p">(</span><span class="ss">attribute: </span><span class="n">value</span><span class="p">)</span> <span class="c1">#or any other thread-safe codes</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="no">Parallel</span><span class="p">.</span><span class="nf">each</span><span class="p">(</span><span class="no">Model</span><span class="p">.</span><span class="nf">find_in_batches</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">array_of_instances</span><span class="o">|</span>
<span class="n">worker</span><span class="p">(</span><span class="n">array_of_instances</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">reconnect!</span></code></pre></figure>
<hr />
<p>By the way, wanna go multi-threading in the browser side too? If you got a CPU intensive JavaScript to run, please utilize <a href="http://www.w3schools.com/html/html5_webworkers.asp">HTML5 Web Workers</a> to free up browsers!</p>
<hr />
<p>Wanna more parallel jobs? How about speed up your RSpect by running parallel on multiple CPU cores? Take a look of <a href="https://github.com/grosser/parallel_tests">parallel_tests gem</a>.</p>
Babun: the new Cygwin for Ruby, Rails, Sinatra and Node.js2015-04-14T00:00:00+00:00https://xjlin0.github.io/tech/2015/04/14/babun-the-new-cygwin-for-ruby-rails-sinatra-and-nodejs
<h3 id="now-you-can-easily-develop-rails-apps-on-ms-windows-like-unix">Now you can easily develop Rails apps on MS Windows like unix<img src="/assets/imgs/baboon.jpeg" alt="baboon" width="20%" /></h3>
<p>It’s hard to setup POSIX compatible environment on MS windows PC for your coding setup, here is the way I found feasible, on Windows 7 64 bit with Babun 1.2.0-dist, updated on Sep 27 2015.</p>
<p>##Installation steps:</p>
<ul>
<li>
<p>Download Babun distribution from <a href="http://babun.github.io">Babun official site</a> by hitting the big <code>[Download Now](http://projects.reficio.org/babun/download)</code> button on the top of the landing page and extract it.</p>
</li>
<li>
<p>Enter MS Windows command line mode (<a href="https://www.youtube.com/watch?v=JOrY5PEo-iE">a.k.a cmd</a>) and go to the extracted folder to start. I recommend to install it under the top folder of a drive to avoid hitting the max path length of the MS Windows file system. Remember, the slash of MS Windows and Unix are different.</p>
</li>
</ul>
<figure class="highlight"><pre><code class="language-console" data-lang="console"><span class="go">install.bat /t "c:/"</span></code></pre></figure>
<p>Babun system will finish the installation within a few minutes, and bring you its default terminal under zsh. One of my favorite things is X-window style mouse copying/pasting, <em><strong>in Babun the mouse selected text in the terminal is immediately copied, you can then paste by mouse right-clicking</strong></em>.</p>
<hr />
<h3 id="quick-and-short-steps-for-starters">Quick and short steps for starters</h3>
<p>If you don’t have time and need Ruby and Node.js now (especially for phase 0 DBC students), just execute the following commands to have Ruby 2.0, Rails 4.0, the latest Sinatra and Node 0.4.12. It’s enough to cover the entire DBC phase 0~2. Under Babun terminal, please enter the following three lines of commands:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">babun update
pact <span class="nb">install </span>ruby ruby-nokogiri ruby-rails ruby-pg libpq-devel libxml2-devel libxslt-devel gcc-g++ patch sqlite3 postgresql curl wget libgmp-devel
gem <span class="nb">install </span>pg sinatra shotgun rspec rails</code></pre></figure>
<p>(Note: Some gems, such as Puma or Turbolinks are not compatible with these settings as of now. Solution will be provided if there’s a working one.)</p>
<p>As of Sep 2015, Rails 4.2.4 and Ruby 2.2.2 should be installed flawlessly by the commands described above. If you unfortunately got some errors about nokogiri error, or you want more updated version of Rails (>4.2.4), or need Heroku tool belt later (say, phase 3), here is what to do:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">gem <span class="nb">install </span>nokogiri <span class="nt">--</span> <span class="nt">--use-system-libraries</span>
gem <span class="nb">install </span>rails
wget <span class="nt">-qO-</span> https://toolbelt.heroku.com/install.sh | sh
<span class="nb">echo</span> <span class="s1">'PATH="/usr/local/heroku/bin:$PATH"'</span> <span class="o">>></span> ~/.zshrc</code></pre></figure>
<p>Next let’s setup node.js. Please download the latest installer for windows at <a href="https://nodejs.org">node official site</a> (v4.1.1 for win64 as the example here), install it. Close and relaunch Babun shell, node REPL console should be available by the following commend.</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">node <span class="nt">-i</span></code></pre></figure>
<p>That’s it! enjoy your free, full-functional POSIX environment under MS windows without deleting any of your files or repartitioning your drive, in minutes.
___</p>
<h2 id="installation-of-rbenv-and-ruby-22">Installation of rbenv and Ruby 2.2</h2>
<p><a href="https://github.com/sstephenson/rbenv">rbenv</a> is a fantastic Ruby version manager, but its installation, with building plugins, requires several steps. To make it simpler, a super great tool, <a href="http://getrbenv.com">getrbenv installer</a>, was developed. However its current master might need a subtle update to fit zsh used in Babun. Here is my workaround to install rbenv with ruby-build and rbenv-update plugin painlessly. Let’s first start without Ruby version assigned. (The following command is ONE line)</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">curl <span class="nt">-sSL</span> https://raw.githubusercontent.com/xjlin0/getrbenv-installer/feature/fix_zshrc_install_dir_pathfix/bin/install.sh | bash <span class="nt">-s</span> <span class="nt">--</span> <span class="nt">--plugins</span> sstephenson/ruby-build,rkh/rbenv-update</code></pre></figure>
<p>Success? Please close Babun shell and restart Babun to get the path works. Now how do you install newer version of Ruby? For Ruby 2.2.3, just need to type:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">rbenv <span class="nb">install </span>2.2.3
rbenv rehash
rbenv global 2.2.3</code></pre></figure>
<p>You can also install older Ruby 2.1.7 or 1.9.3 by <code class="language-plaintext highlighter-rouge">rbenv install 2.1.7</code> or <code class="language-plaintext highlighter-rouge">rbenv install 1.9.3-p551</code>. How about 2.0.0-p647? Currently there is <a href="https://bugs.ruby-lang.org/issues/11065">a naming bug</a> in some versions of Ruby under ext/-test- folder and may stop the compilation in installation. Thus <a href="/assets/imgs/uutoa_printf.patch">I made a patch</a> based on <a href="https://github.com/babun/babun/issues/93">dmattes’s suggestion</a> to prevent the compilation stopping you at printf.c. The patch won’t hurt ruby’s function since it’s in test folder.</p>
<figure class="highlight"><pre><code class="language-csh" data-lang="csh"> curl http://xjlin0.github.io/assets/imgs/uutoa_printf.patch | rbenv install --patch 2.0.0-p647</code></pre></figure>
<hr />
<h2 id="compile-your-own-nodejs-not-recommended">Compile your own Node.js (NOT recommended)</h2>
<p>The windows native binaries by node.js installer should be good for use. However here is some other non-required adventure for those likes experimenting. The last version of Node.js with official support for Cygwin was 0.4.12, according to <a href="https://github.com/babun/babun/issues/216">Lukasz P</a> and <a href="https://github.com/joyent/node/wiki/Installation#building-on-cygwin">Joyent’s suggestion</a>. If you enjoy waiting for compile, here is how to compile and install the old faithful Node.js 0.4.12.</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">wget http://nodejs.org/dist/node-v0.4.12.tar.gz
<span class="nb">tar </span>xvfz node-v0.4.12.tar.gz
<span class="nb">cd </span>node-v0.4.12/
./configure
make
make <span class="nb">install</span></code></pre></figure>
<p>How about newer version? Well, <a href="https://github.com/joyent/node/issues/1734">bnoordhuis’s patch</a> works for v0.5.8. I <a href="/assets/imgs/node-v0.5.8-patched4cygwin.tar.gz">applied the patch and attach here for you to download/install</a>. I have not found ways to compile newer versions.</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">wget http://xjlin0.github.io/assets/imgs/node-v0.5.8-patched4cygwin.tar.gz
<span class="nb">tar </span>xvfz node-v0.5.8-patched4cygwin.tar.gz
<span class="nb">cd </span>node-v0.5.8/
./configure
make
make <span class="nb">install</span></code></pre></figure>
<hr />
<h3 id="some-tips-for-postgresql-service-under-babun--cygwin">Some tips for PostgreSQL service under Babun / Cygwin</h3>
<p>PostgreSQL service is slow but still works under Babun if you follow the following steps to start the service. Basically these are just the steps described in /usr/share/doc/Cygwin/postgresql.README, and you may want to include /usr/sbin in the path.</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">cygrunsrv <span class="nt">-S</span> cygserver
/usr/sbin/initdb.exe <span class="nt">-D</span> /usr/share/postgresql/data
/usr/sbin/pg_ctl start <span class="nt">-D</span> /usr/share/postgresql/data <span class="nt">-l</span> /var/log/postgresql.log
createdb</code></pre></figure>
<p>You may now use psql to check your PostgreSQL service. In case you want to stop the PostgreSQL service someday:</p>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">/usr/sbin/pg_ctl <span class="nt">-D</span> /usr/share/postgresql/data <span class="nt">-l</span> logfile stop
cygrunsrv <span class="nt">-E</span> cygserver</code></pre></figure>
<p>And you may also want to remove the data (/usr/share/postgresql/data) when the PostgreSQL service no longer needed. Please be noted that by such method, even cygserver running is persistent in windows, the closing and restarting of Babun shell will terminate postgresql service so you may need to manually restart it every time.</p>
Choosing between devise and clearance2015-03-20T00:00:00+00:00https://xjlin0.github.io/tech/2015/03/20/choosing-between-devise-and-clearance
<h3 id="these-gems-can-really-help-your-rails-app-on-user-control-">These gems can really help your rails app on user control <img src="/assets/imgs/users.jpg" alt="major incrediable roles" width="20%" /></h3>
<hr />
<table>
<thead>
<tr>
<th>Gem names</th>
<th><strong><em><a href="https://github.com/plataformatec/devise">plataformatec’s_Devise</a></em></strong></th>
<th><strong><em><a href="https://github.com/thoughtbot/clearance">thoughtbot’s Clearance</a></em></strong></th>
</tr>
</thead>
<tbody>
<tr>
<td>Summary</td>
<td>MVC/10modules&more extentions</td>
<td>small & simple</td>
</tr>
<tr>
<td>Install</td>
<td>rails generate devise:install</td>
<td>rails generate clearance:install</td>
</tr>
<tr>
<td> </td>
<td>rails generate devise MODEL</td>
<td>(create migration for you)</td>
</tr>
<tr>
<td>other Auth</td>
<td>by OmniAuth</td>
<td>nil</td>
</tr>
<tr>
<td>mailComfirm</td>
<td>bkg ActiveJob/ActionMailer</td>
<td>bkg (<a href="http://edgeapi.rubyonrails.org/classes/ActionMailer/MessageDelivery.html#method-i-deliver_later">deliver_later/ActionMailer</a>)</td>
</tr>
<tr>
<td>PassWdReset</td>
<td>Recoverable&Confirmable</td>
<td>Yes</td>
</tr>
<tr>
<td>Session</td>
<td>cookie Timeoutable&Rememberable</td>
<td>Yes (by Rack session)</td>
</tr>
<tr>
<td>Sign-up</td>
<td>Validatable & Lockable</td>
<td>SignInGuard</td>
</tr>
<tr>
<td>I18n</td>
<td>yes</td>
<td>yes(even path names)</td>
</tr>
<tr>
<td>Track/log</td>
<td>Yes</td>
<td>nil</td>
</tr>
<tr>
<td>testing</td>
<td>Test helpers</td>
<td>yes with factory_girl_rails</td>
</tr>
<tr>
<td> </td>
<td> </td>
<td>(rails generate clearance:specs)</td>
</tr>
</tbody>
</table>
<hr />
<h3 id="devise">Devise</h3>
<p><a href="http://guides.railsgirls.com/devise/">Here is a great tutorial of Devise can be found on Ruby girls</a>. <a href="http://railsapps.github.io/rails-composer/">Or you can use Rais Composer to install Devise on a new Rails project</a> by the command <code>rails new myapp -m https://raw.github.com/RailsApps/rails-composer/master/composer.rb</code>. Alternatively, you can <a href="http://activeadmin.info">install ActiveAdmin gem</a>, which will automatically install Devise for you.</p>
<h3 id="clearance-heres-how-i-setup-it">Clearance: Here’s how I setup it</h3>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh">rails new clearance <span class="nt">-B</span>
<span class="nb">echo</span> <span class="s2">"gem 'clearance'"</span> <span class="o">>></span> clearance/Gemfile
<span class="nb">echo</span> <span class="s2">"gem 'bcrypt'"</span> <span class="o">>></span> clearance/Gemfile
<span class="nb">echo</span> <span class="s2">"gem 'faker'"</span> <span class="o">>></span> clearance/Gemfile
<span class="nb">cd </span>clearance <span class="o">&&</span> bundle <span class="nb">install
</span>rails generate scaffold User username email password
<span class="nb">echo</span> <span class="s2">"10.times { User.create(username: Faker::Internet.user_name, email: Faker::Internet.email, password: '1') }"</span> <span class="o">>></span> db/seeds.rb
rake db:migrate
rails generate clearance:install
rake db:migrate
rails generate mailer UserMailer
rake db:migrate
rake db:seed
open http://localhost:3000/users
rails s
clear
<span class="nb">echo</span> <span class="s1">'Please patch config and then db migrate before restart rails server'</span></code></pre></figure>
<p><a href="http://blog.anupamsg.me/2012/02/14/enabling-postfix-for-outbound-relay-via-gmail-on-os-x-lion-11/">Don’t forget to change local mailing setting if it’s on localhost</a></p>
<pre>
Clearance post installation
*******************************************************************************
Next steps:
1. Configure the mailer to create full URLs in emails:
# config/environments/{development,test}.rb
config.action_mailer.default_url_options = { host: 'localhost:3000' }
In production it should be your app's domain name.
2. Display user session and flashes. For example, in your application layout:
<% if signed_in? %>
Signed in as: <%= current_user.email %>
<%= button_to 'Sign out', sign_out_path, method: :delete %>
<% else %>
<%= link_to 'Sign in', sign_in_path %>
<% end %>
<div id="flash">
<% flash.each do |key, value| %>
<div class="flash <%= key %>"><%= value %></div>
<% end %>
</div>
3. Migrate:
rake db:migrate
*******************************************************************************
</pre>
The great questions efficiently lead you to the true solution2015-01-31T00:00:00+00:00https://xjlin0.github.io/cultural/2015/01/31/the-great-questions-efficiently-lead-you-to-the-true-solution
<h3 id="theres-no-stupid-questions">there’s no stupid questions<img src="/assets/imgs/q2.jpg" alt="QuestionMark" width="20%" /></h3>
<p>In my previous profession asking questions to address issues or enigma is critical. Right questions leads to the right answers and sometimes the working solution. Keys to the truth are always hidden in our eyes. Asking questions is just like poking around and one successful poking can point to the path to the solution. Sort of like asking questions containing keywords in adventure games.</p>
<p>How can we ask efficient questions to find solutions? First we should isolate the issues needed to be addressed. In programming world sometimes we can create minimal, complete (sufficient), and verifiable (reproducible) example codes. It will help us to ask specific questions about its details. If the isolated parts are too huge to poke around, we should break it into smaller workable problems and continue to research. By following up the answers to the initial question, the chance getting right questions will go higher.</p>
<p>We shouldn’t think a question “stupid”, there’s no such things called stupid questions or jokey questions, since human beings are always subjective and limited by our own imagination. Questions might seems to be relative or irrelative only in our concept, but the true situation may be totally different in reality. When Novoselov and Geim asked themselves “could we peel of layers just by the scotch tape on our desk?”, that might sound stupid and irrelative to their subject of “graphene”, but eventually it efficiently lead to the solution and the 2010 Nobel Prize in physics.</p>
The journey of static page generator2015-01-30T00:00:00+00:00https://xjlin0.github.io/tech/2015/01/30/the-journey-of-static-page-generator
<h3 id="writing-blog-in-a-different-style-">Writing blog in a different style <img src="/assets/imgs/game.png" alt="TheIncrediblesGame" width="20%" /></h3>
<p>Making blogs with manually starting from scratch is great for learning HTML/CSS, however, doing it for every article would be tedious and distracting from writing content. My site is hosted on Github, so I began to search the theme that can be applied without site migration. Many frameworks are outstanding: Jekyll, Bootstrap, etc, so there’s no need to reinvent wheels.</p>
<p>I spent a lot of time with Skinny-bones theme, a pretty Jekyll single pager. However, even after fixing yaml parsing, due to a minor bug released just before the Christmas, Jekyll got some problems to locally build pages successfully. Learned some Jekyll, I then switch to Jekyll-Bootstrap, an easy starter package. It work at my first try, so simple and I wonder why it stopped development.</p>
<p>After applying the Yuya Saito’s the_program theme I moved all my blog over the new platform. Apparently I still need to tweak the jekyll yaml config to set variables. Jekyll-Bootstrap scaffold conventionally use rake to create posts and pages, so input in Markdown and HTML are both accepted. But some of my previous layout got whacked, to name a few, image/assets path, sprites, tabs ( my previous blog used <code class="language-plaintext highlighter-rouge"><dd></code> and <code class="language-plaintext highlighter-rouge"><blockquote></code> for indentation), search function and Github buttons. I also got the problems of pushing to Github due to the forking of different repo, and had to remove my original blog repo completely and recreate. But finally the theme/Jekyll/rake all worked locally and remotely, and here is my first new post on the renewed blog.</p>
<p>All of my cohurt mates are great for making Github pages. For example, in the manually crafting stage, Lukas(great CSS design) and Yumiko(external storage by myjason.com) already baked cool layout. Now in the middle of revision, Ty(Whitespace theme for Octopress, which Chase also used), Evan(Landing Page theme for Start Bootstrap) and Sam(Bootstrap+Prism.js) all made impressive works.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1. tried several times with skinny bones starter but failed.
2. tried with A-Sam's solution didnt' work http://blog.ssyog.com/blog/jekyll/install-jekyll-based-on-github-pages.html Doens't even have jekyll commands after gem install jekyll?
3. tried 3 steps and didn't work http://jekyllrb.com/
4. run brew doctor and found I have not update for several month.
5. tried again with jekyllrb.com and work by "jekyll new blog" and "jekyll serve --baseurl '/blog'"
http://127.0.0.1:4000/blog/jekyll/update/2015/01/28/welcome-to-jekyll.html
6. found I might download a differnt source of zip file of skinny bones in step 1.
7. repeat step 1 and copy welcome markdown file to _posts folder under step 6/blog
8. edit _config.yml and found my mis-swap filename for line 5&7. and I may have misunderstand the url & baseurl part. Realized I might failed again and need to go back to vanilla installation of jekyll in step 3 to get most basic _config.yml to restart.
9. still getting errors of running skinny bones, google "cannot load such file -- jekyll/version" and got the answer "gem cleanup". Run bundle install again after clean up
https://github.com/jekyll/jekyll/issues/3084
10. can't work it out. Give up on skinny bones theme
11. Tried The program theme of Jekyll-Bootstrap got the same problems.
12 found there are many gems are installed by root permission
/Users/j/.rbenv/versions/2.0.0-p481/lib/ruby/gems/2.0.0/gems/
13. remove all gems and reinstall jekyll with "gem install jekyll --user-install" and "bundle install --path /Users/j/.gem/ruby/2.0.0/gems"
14.trying the syntax of yaml. "" every thing in the _config.yml ==> found the problem was _config.yml
15.copy everything from basic yml to skinny bones yml ==> it's the tab/space before the key in the siteowner part that yml complains! (testing by irb -ryaml and YAML.load_file 'filename.yml' )
16 install octopress for publishing
17 octopress new post "testAgain" --dir test to get _posts/test/2015-01-28-testagain.markdown
bundle exec jekyll build
bundle exec jekyll serve
========================Switch to Jekyll Bootstrap=====
http://jekyllbootstrap.com/usage/jekyll-quick-start.html
=====================================================
under username.github.io
A. git clone https://github.com/plusjade/jekyll-bootstrap.git blog
B. cd blog
C. jekyll serve #==> it works!!! (even with Jekyll 2.5.3)
D. first attemp to install the program theme failed
rake theme:install git="https://github.com/jekyllbootstrap/theme-the-program.git"
/Users/j/.rbenv/versions/2.0.0-p481/bin/rake:23:in `load': cannot load such file -- /Users/j/.rbenv/versions/2.0.0-p481/lib/ruby/gems/2.0.0/gems/rake-0.9.6/bin/rake (LoadError)
from /Users/j/.rbenv/versions/2.0.0-p481/bin/rake:23:in `<main>'
E.gem install rake #==> install rake-10.4.2
F. repeat D and it works locally!!!!!!!!!!!!!!!!!!!!!!!!!
G. found it take <img> tags for proper image positioning.
H. change css under asset/ works locally
http://jekyllbootstrap.com/api/theme-api.html
I. found I can't push to github.
H. remove my repo on Github and redo install following fresh repo
http://jekyllbootstrap.com/usage/jekyll-quick-start.html
error:
xjlin0.github.io [master] :> git remote -v
origin git@github.com:xjlin0/xjlin0.github.io.git (fetch)
origin git@github.com:xjlin0/xjlin0.github.io.git (push)
xjlin0.github.io [master] :> git push origin master
Permission denied (publickey).
xjlin0.github.io [master] :> git remote set-url origin https://github.com/xjlin0/xjlin0.github.io.git #==>works for push
I. yaml working format for liquid syntax {{ site.author.name }}
author : (left blank)
name : whatever_name
email : whatever_email
_includes/themes/the-program/default.html
J. rake post title="Hello World" or rake page name="about.md"
K. jekyll serve #to start local server for browsing
</code></pre></div></div>
Anger Management2015-01-21T00:00:00+00:00https://xjlin0.github.io/cultural/2015/01/21/anger-management
<h3 id="what-happened-to-me-when-upset">what happened to me when upset<img src="/assets/imgs/conflict.jpg" alt="JackJackSupercharged" width="20%" /></h3>
<div id="article">
<p>Not long ago I spent a lot of time showing someone at the work how to do a complicate process which requires hours to get correct results. I spent lot of time to explain and work with him, but he repeatedly argued a setting that the boss already decided, and I can't change that at my position. Around midnight we haven't finished but again he tried to change that setting. I fed up and tell him to finish that process alone and left. But later on the result was wrong and unhappy.</p>
<p>The source of my conflict feeling was, I put so much time trying to make this process smoothly and correctly, however, he still want to alter a setting which I was not authorized to change. I got upset and don't want to be involved for any possible resulting errors. I didn't take any actions and just left, which appear to be not the best solution. Because it's not the best idea to call the boss at midnight for non emergent events, I should have plainly tell him the reason behind and offer him the chance to talking to boss and decide later to prevent such error.</p>
<p>What I learned from this case was: I am not that capable of handling my emotion, just like when I facing the intense situation in the pairing session. If a situation occurs, I should filter out my feeling, maybe by taking a break, frankly discuss with the other person, and offer a better solution to solve the situation. Thanks DBC for helping to find myself, maybe I will learn and change soon.</p>
</div>
Cache the results to skip repetitive calculations2015-01-19T00:00:00+00:00https://xjlin0.github.io/tech/2015/01/19/cache-the-results-to-skip-repetitive-calculations
<h3 id="memoization-for-the-functional-programming">Memoization for the functional programming<img src="/assets/imgs/megamemory.jpg" alt="BabyFacingKariFocused" width="20%" /></h3>
<div id="article">
<p><a href="http://en.wikipedia.org/wiki/Memoization#Functional_programming">Memoization</a>, storing something in the memory, is very important in the functional programming to avoid repetitive calculation. Such process, sometimes can be used as the cache mechanism, can save a lot of time, since the memory can be used as containers for expansive calculations or slow I/O results. Yesterday I read <a href="http://www.quora.com/What-are-some-cool-Ruby-tricks">a magic Ruby trick to realize the memoization</a> and would like to try here:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="no">Welcome</span> <span class="n">to</span> <span class="no">IRB</span><span class="o">.</span> <span class="no">You</span> <span class="n">are</span> <span class="n">using</span> <span class="n">ruby</span> <span class="mf">2.0</span><span class="o">.</span><span class="mi">0</span><span class="n">p576</span> <span class="p">(</span><span class="mi">2014</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">19</span><span class="p">)</span> <span class="p">[</span><span class="n">i386</span><span class="o">-</span><span class="n">cygwin</span><span class="p">]</span><span class="o">.</span> <span class="no">Have</span> <span class="n">fun</span> <span class="p">;)</span>
<span class="o">>></span> <span class="n">fact</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span><span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="p">,</span><span class="n">k</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span> <span class="n">k</span><span class="o"><=</span><span class="mi">1</span> <span class="p">?</span> <span class="mi">1</span> <span class="p">:</span> <span class="n">k</span> <span class="o">*</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="p">)</span> <span class="p">}</span>
<span class="o">>></span> <span class="n">fact</span><span class="p">[</span><span class="mi">400</span><span class="p">]</span> <span class="c1">#=> instant result of factorial!</span>
<span class="o">>></span> <span class="n">fibo</span> <span class="o">=</span> <span class="no">Hash</span><span class="p">.</span><span class="nf">new</span><span class="p">{</span> <span class="o">|</span><span class="n">h</span><span class="p">,</span><span class="n">k</span><span class="o">|</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="p">]</span> <span class="o">=</span> <span class="p">(</span> <span class="n">k</span><span class="o"><=</span><span class="mi">2</span> <span class="p">?</span> <span class="mi">1</span> <span class="p">:</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="o">-</span><span class="mi">2</span><span class="p">]</span> <span class="o">+</span> <span class="n">h</span><span class="p">[</span><span class="n">k</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="p">)</span> <span class="p">}</span>
<span class="o">>></span> <span class="n">fibo</span><span class="p">[</span><span class="mi">500</span><span class="p">]</span> <span class="c1">#=> instant result of Fibonacci !</span>
<span class="o">>></span> <span class="k">def</span> <span class="nf">fib</span><span class="p">(</span><span class="n">n</span><span class="p">)</span> <span class="c1"># let's compare with recursive</span>
<span class="k">return</span> <span class="mi">1</span> <span class="k">if</span> <span class="n">n</span> <span class="o"><=</span> <span class="mi">2</span>
<span class="n">fib</span><span class="p">(</span> <span class="n">n</span> <span class="o">-</span> <span class="mi">1</span> <span class="p">)</span> <span class="o">+</span> <span class="n">fib</span><span class="p">(</span> <span class="n">n</span> <span class="o">-</span> <span class="mi">2</span> <span class="p">)</span>
<span class="o">>></span> <span class="k">end</span>
<span class="o">>></span> <span class="n">fib</span><span class="p">(</span><span class="mi">500</span><span class="p">)</span> <span class="c1">#=> This will probably freeze your computer, please don't try it.</span></code></pre></figure>
<p>See, when the new hash is declared here, the hash value is calculated only to the previous 1 or 2 elements. Since most of the calculation results are stored in the hash, the complexity of calculating Fibonacci here is only <code>O(n)</code>, or even less if repeat. However, using recursive to to calculate Fibonacci will get much higher <code>O(2^n)</code> for each calling, what a difference! Another cool way is to utilize Ruby's <code>Fiber</code> class since it 'freezes' upon yield, shown in Mat's <a href="https://books.google.com/books?id=jcUbTcr5XWwC">The Ruby Programming Language</a></p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="no">Welcome</span> <span class="n">to</span> <span class="no">IRB</span><span class="o">.</span> <span class="no">You</span> <span class="n">are</span> <span class="n">using</span> <span class="n">ruby</span> <span class="mf">2.0</span><span class="o">.</span><span class="mi">0</span><span class="n">p576</span> <span class="p">(</span><span class="mi">2014</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">19</span><span class="p">)</span> <span class="p">[</span><span class="n">i386</span><span class="o">-</span><span class="n">cygwin</span><span class="p">]</span><span class="o">.</span> <span class="no">Have</span> <span class="n">fun</span> <span class="p">;)</span>
<span class="o">>></span> <span class="n">fib</span> <span class="o">=</span> <span class="no">Fiber</span><span class="p">.</span><span class="nf">new</span> <span class="k">do</span>
<span class="o">>></span> <span class="n">x</span><span class="p">,</span> <span class="n">y</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">1</span>
<span class="o">>></span> <span class="kp">loop</span> <span class="k">do</span>
<span class="o">>></span> <span class="no">Fiber</span><span class="p">.</span><span class="nf">yield</span> <span class="n">y</span>
<span class="o">>></span> <span class="n">x</span><span class="p">,</span><span class="n">y</span> <span class="o">=</span> <span class="n">y</span><span class="p">,</span><span class="n">x</span><span class="o">+</span><span class="n">y</span>
<span class="o">>></span> <span class="k">end</span>
<span class="o">>></span> <span class="k">end</span>
<span class="o">>></span>
<span class="o">>></span> <span class="mi">500</span><span class="p">.</span><span class="nf">times</span> <span class="p">{</span> <span class="nb">puts</span> <span class="n">fib</span><span class="p">.</span><span class="nf">resume</span> <span class="p">}</span> <span class="c1">#=> instant result of Fibonacci!</span></code></pre></figure>
<p>Of course we can implement the memoization in JavaScript. Here are the same factorial and Fibonacci using memoization, by the same concept:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="nx">node</span> <span class="nx">v0</span><span class="p">.</span><span class="mf">10.13</span> <span class="nx">console</span>
<span class="o">></span> <span class="kd">var</span> <span class="nx">facto</span> <span class="o">=</span> <span class="p">{</span> <span class="mi">0</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">:</span><span class="mi">1</span> <span class="p">};</span>
<span class="o">></span> <span class="kd">function</span> <span class="nx">fac</span><span class="p">(</span><span class="nx">n</span><span class="p">){</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">facto</span><span class="p">[</span><span class="nx">n</span><span class="p">])</span> <span class="nx">facto</span><span class="p">[</span><span class="nx">n</span><span class="p">]</span> <span class="o">=</span> <span class="nx">n</span> <span class="o">*</span> <span class="nx">fac</span><span class="p">(</span><span class="nx">n</span><span class="o">-</span><span class="mi">1</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">facto</span><span class="p">[</span><span class="nx">n</span><span class="p">]</span> <span class="p">}</span>
<span class="o">></span> <span class="nx">fac</span><span class="p">(</span><span class="mi">170</span><span class="p">)</span> <span class="c1">// instant result of factorial!</span>
<span class="o">></span> <span class="kd">var</span> <span class="nx">fibon</span> <span class="o">=</span> <span class="p">{</span> <span class="mi">0</span><span class="p">:</span><span class="mi">0</span><span class="p">,</span> <span class="mi">1</span><span class="p">:</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">:</span><span class="mi">1</span><span class="p">};</span>
<span class="o">></span> <span class="kd">function</span> <span class="nx">fib</span><span class="p">(</span><span class="nx">n</span><span class="p">){</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">fibon</span><span class="p">[</span><span class="nx">n</span><span class="p">])</span> <span class="nx">fibon</span><span class="p">[</span><span class="nx">n</span><span class="p">]</span> <span class="o">=</span> <span class="nx">fib</span><span class="p">(</span><span class="nx">n</span><span class="o">-</span><span class="mi">2</span><span class="p">)</span> <span class="o">+</span> <span class="nx">fib</span><span class="p">(</span><span class="nx">n</span><span class="o">-</span><span class="mi">1</span><span class="p">);</span>
<span class="k">return</span> <span class="nx">fibon</span><span class="p">[</span><span class="nx">n</span><span class="p">]</span> <span class="p">}</span>
<span class="o">></span> <span class="nx">fib</span><span class="p">(</span><span class="mi">170</span><span class="p">)</span> <span class="c1">// instant result of Fibonacci!</span></code></pre></figure>
<p>What does it compare to conventional looping method? You would think? Well, for new calculations, a looping method calculating Fibonacci also gets the complexity of <code>O(n)</code> as well, but if more calculations on the way, the stored results in the memory will function as the <strong><em>cache</em></strong> and less processor power or time will be needed, compared to the looping without memoization shown below.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"> <span class="nx">node</span> <span class="nx">v0</span><span class="p">.</span><span class="mf">10.13</span> <span class="nx">console</span>
<span class="o">></span> <span class="kd">function</span> <span class="nx">fibl</span><span class="p">(</span><span class="nx">n</span><span class="p">){</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">n</span> <span class="o"><</span> <span class="mi">3</span><span class="p">)</span> <span class="k">return</span> <span class="mi">1</span><span class="p">;</span>
<span class="kd">var</span> <span class="nx">last1</span> <span class="o">=</span> <span class="mi">1</span><span class="p">,</span> <span class="nx">last2</span> <span class="o">=</span> <span class="mi">0</span><span class="p">,</span> <span class="nx">sum</span> <span class="o">=</span> <span class="mi">0</span><span class="p">;</span>
<span class="k">for</span> <span class="p">(</span><span class="kd">var</span> <span class="nx">i</span><span class="o">=</span><span class="mi">2</span><span class="p">;</span> <span class="nx">i</span> <span class="o"><</span> <span class="nx">n</span><span class="o">+</span><span class="mi">1</span><span class="p">;</span> <span class="nx">i</span><span class="o">++</span><span class="p">){</span>
<span class="nx">sum</span> <span class="o">=</span> <span class="nx">last1</span> <span class="o">+</span> <span class="nx">last2</span><span class="p">,</span> <span class="nx">last2</span> <span class="o">=</span> <span class="nx">last1</span><span class="p">,</span> <span class="nx">last1</span> <span class="o">=</span> <span class="nx">sum</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nx">sum</span><span class="p">;</span> <span class="p">}</span>
<span class="o">></span> <span class="nx">fibl</span><span class="p">(</span><span class="mi">170</span><span class="p">)</span> <span class="c1">// calculation result of Fibonacci without the cache.</span></code></pre></figure>
</div>
Inheritance in Ruby, Python and JavaScript2015-01-14T00:00:00+00:00https://xjlin0.github.io/tech/2015/01/14/inheritance-in-ruby-python-and-javascript
<h3 id="oop-in-three-languages-">OOP in three languages <img src="/assets/imgs/iclass.jpg" alt="BabyFacingKariFocused" width="20%" /></h3>
<div id="article">
<p>Ruby is a class-based language, thus it is straightforward to create objects in classes for inheritance as described in my previous posts. However, it takes some steps to mimic classes in JavaScript, which is a prototype-based language. Let's do some comparison. Here is how we do inheritance in Ruby:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"> <span class="no">Welcome</span> <span class="n">to</span> <span class="no">IRB</span><span class="o">.</span> <span class="no">You</span> <span class="n">are</span> <span class="n">using</span> <span class="n">ruby</span> <span class="mf">2.0</span><span class="o">.</span><span class="mi">0</span><span class="n">p576</span> <span class="p">(</span><span class="mi">2014</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">19</span><span class="p">)</span> <span class="p">[</span><span class="n">i386</span><span class="o">-</span><span class="n">cygwin</span><span class="p">]</span><span class="o">.</span> <span class="no">Have</span> <span class="n">fun</span> <span class="p">;)</span>
<span class="o">>></span> <span class="k">class</span> <span class="nc">Supermen</span>
<span class="k">def</span> <span class="nf">fast_moving</span>
<span class="s2">"Shuuooo ..........you just moved!"</span>
<span class="k">end</span>
<span class="o">>></span> <span class="k">end</span>
<span class="o">>></span> <span class="k">class</span> <span class="nc">Incredibles</span> <span class="o"><</span> <span class="no">Supermen</span>
<span class="k">def</span> <span class="nf">wear_uniform</span>
<span class="s2">"The red and black looked so nice!"</span>
<span class="k">end</span>
<span class="o">>></span> <span class="k">end</span>
<span class="c1"># note: inheritance can also be done when initiating a new instance of Class</span>
<span class="c1"># Incredibles = Class.new(Supermen)</span>
<span class="o">>></span> <span class="n">clark</span> <span class="o">=</span> <span class="no">Superman</span><span class="p">.</span><span class="nf">new</span>
<span class="o">>></span> <span class="n">clark</span><span class="p">.</span><span class="nf">fast_moving</span> <span class="c1">#=> "Shuuooo ..........you just moved!"</span>
<span class="o">>></span> <span class="n">clark</span><span class="p">.</span><span class="nf">wear_uniform</span> <span class="c1">#=> NoMethodError.</span>
<span class="o">>></span> <span class="n">rob</span> <span class="o">=</span> <span class="no">Incredibles</span><span class="p">.</span><span class="nf">new</span>
<span class="o">>></span> <span class="n">rob</span><span class="p">.</span><span class="nf">fast_moving</span> <span class="c1">#=> "Shuuooo ..........you just moved!"</span>
<span class="o">>></span> <span class="n">rob</span><span class="p">.</span><span class="nf">wear_uniform</span> <span class="c1">#=> "The red and black looked so nice!"</span></code></pre></figure>
<p>In Python, we can create instance by defining custom factory function. If a class is defined, you can create an instance by <code> instance = Class() </code>. Python interpreter will actually call <code>Class().method(instance)</code> and execute method defined in that class. Don't forget to include <code>self</code> in the definition of methods so Python knows which instance to pass for executing the method.
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">Python</span> <span class="mf">2.7</span><span class="p">.</span><span class="mi">9</span> <span class="n">shell</span>
<span class="o">>>></span> <span class="k">class</span> <span class="nc">Supermen</span><span class="p">:</span>
<span class="p">...</span> <span class="k">def</span> <span class="nf">fast_moving</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="p">...</span> <span class="k">return</span> <span class="s">"Shuuooo ..........you just moved!"</span>
<span class="p">...</span>
<span class="o">>>></span> <span class="n">clark</span> <span class="o">=</span> <span class="n">Supermen</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">clark</span><span class="p">.</span><span class="n">fast_moving</span><span class="p">()</span>
<span class="s">'Shuuooo ..........you just moved!'</span>
<span class="o">>>></span> <span class="k">class</span> <span class="nc">Incredibles</span><span class="p">(</span><span class="n">Supermen</span><span class="p">):</span>
<span class="p">...</span> <span class="k">def</span> <span class="nf">wear_uniform</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="p">...</span> <span class="k">return</span> <span class="s">"The red and black looked so nice!"</span>
<span class="p">...</span>
<span class="o">>>></span> <span class="n">rob</span> <span class="o">=</span> <span class="n">Incredibles</span><span class="p">()</span>
<span class="o">>>></span> <span class="n">rob</span><span class="p">.</span><span class="n">wear_uniform</span><span class="p">()</span>
<span class="s">'The red and black looked so nice!'</span>
<span class="o">>>></span> <span class="n">rob</span><span class="p">.</span><span class="n">fast_moving</span><span class="p">()</span>
<span class="s">'Shuuooo ..........you just moved!'</span></code></pre></figure>
<p>In JavaScript a popular way to mimic class is through constructor functions and use new functions to create instances, and inherit such "classes" by the prototype assignment:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"> <span class="nx">node</span> <span class="nx">v0</span><span class="p">.</span><span class="mf">10.13</span> <span class="nx">console</span>
<span class="o">></span> <span class="kd">function</span> <span class="nx">Superman</span><span class="p">(){</span>
<span class="k">this</span><span class="p">.</span><span class="nx">fast_moving</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">Shuuooo ..........you just moved!</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="o">></span> <span class="kd">function</span> <span class="nx">Incredibles</span><span class="p">(){</span>
<span class="k">this</span><span class="p">.</span><span class="nx">wear_uniform</span> <span class="o">=</span> <span class="dl">"</span><span class="s2">The red and black looked so nice!</span><span class="dl">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="o">></span> <span class="nx">Incredibles</span><span class="p">.</span><span class="nx">prototype</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Superman</span><span class="p">();</span>
<span class="o">></span> <span class="kd">var</span> <span class="nx">clark</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Superman</span><span class="p">();</span>
<span class="o">></span> <span class="nx">clark</span><span class="p">.</span><span class="nx">fast_moving</span><span class="p">;</span>
<span class="dl">'</span><span class="s1">Shuuooo ..........you just moved!</span><span class="dl">'</span>
<span class="o">></span> <span class="nx">clark</span><span class="p">.</span><span class="nx">wear_uniform</span><span class="p">;</span>
<span class="kc">undefined</span>
<span class="o">></span> <span class="kd">var</span> <span class="nx">rob</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">Incredibles</span><span class="p">();</span>
<span class="o">></span> <span class="nx">rob</span><span class="p">.</span><span class="nx">fast_moving</span><span class="p">;</span>
<span class="dl">'</span><span class="s1">Shuuooo ..........you just moved!</span><span class="dl">'</span>
<span class="o">></span> <span class="nx">rob</span><span class="p">.</span><span class="nx">wear_uniform</span><span class="p">;</span>
<span class="dl">'</span><span class="s1">The red and black looked so nice!</span><span class="dl">'</span></code></pre></figure>
<p>Functions in JavaScript are special and can be used as classes sometimes. In the above example we start two functions by constructors, and assign prototype to inherit. Therefore Rob can use Superman's methods by such assignment without pre-definition.</p>
<p>However such prototype assignment will result wrong <code>instanceof</code> results. For example, in the above case the return values of <code>rob instanceof Superman</code> and <code>rob instanceof Incredibles</code> would be both true, because the constructor of the Incredibles points to Superman after the prototype assignment. Some people suggest to use <code>Incredibles.prototype.constructor = Incredibles</code> after the prototype assignment to correct the wrong chaining.</p>
</div>
</p></div>
Critical values that matter2015-01-14T00:00:00+00:00https://xjlin0.github.io/cultural/2015/01/14/critical-values-that-matter
<h3 id="whats-important-for-my-life">what’s important for my life?<img src="/assets/imgs/value.jpg" alt="JackJackSupercharged" width="20%" /></h3>
<div id="article">
<p>The world is always changing, and so do our lives. What is the important values that mean a lot to your life? For me, keeps my families happy, getting the sense of accomplishment, having feeling of the confidence, and helping others in the world are all critical to me.</p>
<p>My parents were very busy, so our family get together time is very valuable. I always remember the moments that enjoying all sorts of the things. Now I have my own family and I found myself is also so busy; Déjà vu, right? but such busy life makes my long for happy family life even more.</p>
<p>My instant reaction when interacting with people is slow, instead, I am a planning/executing person. The progress of the thinking & planing, to the end to achieve something in the end, make my life so delightful. Maybe the reason I like such process is, it's a proof of my strategy.</p>
<p>Why would I like to planning ahead and then execute it? I asked myself many times. One other important reason is losing the control of the status is not my favorite. The planning ahead make me feel confident. And such confidence make me progress on my plan steady toward my goals.</p>
<p>Through my religion I also like to help all other people. Helping people to do something good not only light up my life but also elevate people's life. Using my knowledge and power to help people for good, is also a critical value to my life.</p>
</div>
Laughing about Stereotypes2014-12-29T00:00:00+00:00https://xjlin0.github.io/cultural/2014/12/29/laughing-about-stereotypes
<h3 id="lets-joke-and-kill-it">Let’s joke and kill it<img src="/assets/imgs/laugh.jpg" alt="JackJackSupercharged" width="20%" /></h3>
<div id="article">
<p>Human are subjective based on senses, and prone to project resemble images to others alike. Such judgments of people before their inner and true values sometimes get very bad, not just limited to office cultures or seriously negative impacts on productivities, but also eventually the great loss of the entire peaceful society toward Utopia.</p>
<p>Let's close our eyes and think, whatever stereotypes or discriminations based on will eventually become pointless at some levels. Look at kid's fight and we would realize that. What's important is we need to put down these ridiculous idea and work together. My plan is to ease the stereotype threads by over-communication, and eventually to joke myself to get over it, hopefully others would join these trend. After reading at the following stereotypes and think again: why would we needs stereotype threads hurting ourselves?</p>
<p>The following article is not my work. The permission for the English translation from the original article author Vinta is <a href="http://vinta.ws/blog/695#respond">here.</a></p>
<hr />
<h1>The condescending chain of software engineers</h1>
<p>Recently there's a trend for everybody to code around the world, say, even the <a href="http://www.theverge.com/2014/12/9/7358111/barack-obama-coding">American President writes JavaScript</a>. However, as a professional <strike>web surfer</strike> software developer (a more in term for the software engineer nowadays), here I'd like to give you an important advice for all beginner coders: Carefully choose your first programming language.</p>
<p>In the ecology of software engineers (they're called "coding monkeys" or "coding peasants" in China), It's a common culture that programmers condescend to others in the complicated chain in every field of the programming worlds. Topics varies from: programming languages, editors, frameworks, to "should the curly brackets be in the same or next line of if?". Among these topics, "what programming language do you use?" is the hottest one. Therefore, for a brand new beginners, you will be in the lowest level of the condescending chain, if you don't carefully choose your programming language.</p>
<p>How severe and crucial is the condescending chain of software engineers?</p>
<h2>Round I: Programming languages</h2>
<p>Engineers knowing functional programming condescend to engineers talking about software design pattern. Engineers talking about software design pattern condescend to engineers always murmuring "it's not OOP at all!". Engineers always murmuring "it's not OOP at all!" condescend to engineers complaining "Huh? OOP what? couldn't I just write the repetitive codes in a function?". Engineers complaining "Huh? OOP what? couldn't I just write the repetitive codes in a function?" condescend to engineers copying/pasting the same code everywhere. Engineers copying/pasting the same codes everywhere condescend to project managers.</p>
<p>Engineers writing static type-checking systems condescend to engineers writing dynamic type-checking systems.</p>
<p>Assembler programmers condescend to C programmers, C programmers condescend to C++ programmers, C++ programmers condescend to Java and C# programmers, and Java and C# programmers condescend to each other. And C# programmers condescend to Visual Basic programmers and programmers read C# as "C-pound". The programmers read C# as "C-pound" condescend to programmers who think HTML as a programming language.</p>
<p>Engineers using python 3 condescend to engineers still using python 2. Engineers still using python 2 condescend to engineers getting Unicode Encode Error.</p>
<p>iOS programmers condescend to Android programmers. Android programmers condescend to Windows Phone programmers.</p>
<p>Engineers with one year experience in Swift condescend to engineers with five year experience in Objective-C. Objective-C engineers condescend to engineers wrapping native apps by PhoneGap.</p>
<p>React.js programmers condescend to AngularJS programmers. AngularJS programmers condescend to jQuery programmers. jQuery programmers condescend to <a href="http://vanilla-js.com/">Vanilla JavaScript</a> programmers. Vanilla JavaScript programmers condescend to Internet Explorer users.</p>
<p>Engineers using debugger condescend to engineers using assert. Engineers using assert condescend to engineers only using print(); Engineers debugging by console.log() condescend to engineers debugging by alert().</p>
<p>Ruby on Rails engineers condescend to all other engineers. What? Are you talking about Ruby? No, Ruby is just a framework of Ruby on Rails, not any sort of programming languages!</p>
<p><a href="http://www.techug.com/if-programming-languages-were-weapons">All programmers condescend to PHP programmers.</a></p>
<h2>Round II: Tools</h2>
<p>Engineers using text editors condescend to engineers using IDE.</p>
<p>Vim coders condescend to Emacs coders. Emacs coders condescend to Vim coders, both Vim and Emacs coders condescend to coders using any other editors. Coders using Atom, Notepad++, Sublime Text condescend to coders using windows notepad.</p>
<p>Engineers using Android Studio or IntelliJ IDEA condescend to engineers using Eclipse. Engineers using Eclipse condescend to engineers using NetBeans.</p>
<p>Programmers using dark-background editors condescend to programmers using white-background editors.</p>
<p>Engineers indenting by space condescend to engineers indenting by tab. Engineers indenting by tab condescend to engineers indenting by mixing space and tab.</p>
<p>Engineers using Git or Mercurial condescend to engineers using Subversion. Engineers using Subversion condescend to engineers using Dropbox for version control. Engineers using Dropbox for version control condescend to engineers who have absolutely no idea about version control.</p>
<p>Engineers knowing Github condescend to engineers not knowing Github. Engineers who get private repo in Github condescend to those engineers switched to BitBucket for free private repo.</p>
<p>Engineers using Zsh condescend to engineers using Bash. Engineers using Bash condescend to engineers using Cygwin. Engineers using Cygwin condescend to engineers using Windows CMD commend line. Engineers using Windows CMD commend line condescend to engineers using GUI.</p>
<p>Engineers using IRC condescend to engineers using HipChat. Engineers using HipChat condescend to designers using Slack and project managers.</p>
<p>Engineers documenting by reStructuredText condescend to engineers documenting by Markdown. Engineers documenting by Markdown condescend to engineers documenting by HTML. Engineers documenting by HTML condescend to engineers who don't document at all. At last Engineers documenting by LaTeX condescend to all other engineers.</p>
<p>Engineers using Nginx condescend to engineers using Apache. Engineers using Apache condescend to engineers using IIS.</p>
<p>Engineers using Spark condescend to engineers using Hadoop. Engineers using Hadoop condescend to engineers using Hadoop for merely several GB of data. Engineers using Hadoop for processing 1 GB of data condescend to engineers using NoSQL. Engineers using NoSQL condescend to engineers using relational database. Engineers using relational database condescend to project manager using Excel.</p>
<p>Engineers using Docker for server deployment condescend to engineers using Ansible or Puppet for server deployment. Engineers using Ansible or Puppet for server deployment condescend to engineers using Fabric for server deployment. Engineers using Fabric for server deployment condescend to engineers manually deploy servers by SSH.</p>
<h2>Round III: OS</h2>
<p>Engineers using Mac OSX condescend to engineers using Linux. Engineers using Linux condescend to engineers using Windows.</p>
<p>Engineers using Debian despise engineers using Ubuntu. Engineers using Ubuntu despise engineers using non-LTS version of Ubuntu.</p>
<h2>Round IV: Hardware</h2>
<p>Engineers using MacBook Pro Retina condescend to engineers using MacBook Air. Engineers using MacBook Air condescend to engineers using ThinkPad. Then engineers using Raspberry Pi condescend to engineers using MacBook Pro Retina.</p>
<p>Engineers using Dvorak keyboards condescend to engineers using Mac keyboards. Engineers using Mac keyboards condescend to engineers using QWERTY keyboards. Engineers using QWERTY keyboards condescend to engineers not knowing QWERTY keyboards. Engineers not knowing QWERTY keyboards condescend to designers using graphics tablet.</p>
<p>Engineers sitting on Aeron chairs condescend to engineers sitting on ordinary office chairs. Engineers sitting on ordinary office chairs condescend to project managers sitting on the exactly the same ordinary office chairs. Then, standing & coding engineers condescend to engineers sitting on Aeron chairs.</p>
<h2>Round V: Office</h2>
<p>Hardware engineers condescend to software engineers.</p>
<p>Engineers developing OS condescend to engineers developing Web. Engineers developing Web condescend to engineers developing desktop applications.</p>
<p>Back-end engineers condescend to front-end engineers.</p>
<p>Engineers and designers condescend to each other.</p>
<p>Engineers believing Test-Driven Development condescend to engineers make up tests after writing codes. Engineers make up tests after writing codes condescend to engineers not writing tests at all. Engineers not writing tests condescend to project managers who fxxking change specs again.</p>
<p>Engineers without certifications condescend to engineers passed without many certifications.</p>
<p>Engineers wearing business casual condescend to engineers wearing suits and ties. Engineers wearing suits and ties condescend to engineers wearing college shirts.</p>
<p>Engineers laughing at this article condescend to those engineers saying "Programming languages are just tools, there's not meaningful to condescend to each others, every programming languages gets good and bad fitting situations."</p>
<hr />
<p>If you are still not knocked out after reading these crucial condescending chains, I have to remind you the most important things here: Go to get a girl friend first then learn programing. Because once you become a software engineer you won't have a chance to get one.</p>
<p>PS. The original author Vinta welcomes friends with any comments at his sites: <a href="https://github.com/vinta">Github</a>, <a href="https://twitter.com/vinta">Twitter</a> or <a href="https://www.facebook.com/vintalines">Facebook</a>.</p>
</div>
List all shifts of Incredibles2014-12-28T00:00:00+00:00https://xjlin0.github.io/tech/2014/12/28/list-all-shifts-of-incredibles
<h3 id="the-composition-and-inheritance-in-ruby-class">The composition and inheritance in Ruby class<img src="/assets/imgs/family.jpg" alt="BabyFacingKariFocused" width="20%" /></h3>
<p>In the previous blog I showed how to use undefined methods in class by attr_accessor and method_missing. Let's talk more about how to use undefined methods by inheritance and composition since it's useful for the object-oriented programming. Here we define super powers of the Incredibles.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="no">Welcome</span> <span class="n">to</span> <span class="no">IRB</span><span class="o">.</span> <span class="no">You</span> <span class="n">are</span> <span class="n">using</span> <span class="n">ruby</span> <span class="mf">2.0</span><span class="o">.</span><span class="mi">0</span><span class="n">p576</span> <span class="p">(</span><span class="mi">2014</span><span class="o">-</span><span class="mi">09</span><span class="o">-</span><span class="mi">19</span><span class="p">)</span> <span class="p">[</span><span class="n">i386</span><span class="o">-</span><span class="n">cygwin</span><span class="p">]</span><span class="o">.</span> <span class="no">Have</span> <span class="n">fun</span> <span class="p">;)</span>
<span class="o">>></span> <span class="k">class</span> <span class="nc">Supermen</span>
<span class="o">|</span> <span class="k">def</span> <span class="nf">instant_moving</span>
<span class="o">|</span> <span class="s2">"Shuuooo .......... you just moved!"</span>
<span class="o">|</span> <span class="k">end</span>
<span class="o">|</span> <span class="k">def</span> <span class="nf">shield</span>
<span class="o">|</span> <span class="s2">"The invisible wall is now protecting you!"</span>
<span class="o">|</span> <span class="k">end</span> <span class="c1"># you can list many super powers on and on....</span>
<span class="o">|</span> <span class="k">end</span> <span class="c1">#=> nil</span>
<span class="o">>></span> <span class="no">Supermen</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">instant_moving</span> <span class="c1">#=> "Shuuooo .......... you just moved!"</span></code></pre></figure>
<p>However, with great power there must also come great responsibility. Let's define their busy life here, and supermen get super power by the inheritance. Notice the <em><strong>"< Supermen"</strong></em> below: </p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="o">>></span><span class="k">class</span> <span class="nc">Incredibles</span> <span class="o"><</span> <span class="no">Supermen</span>
<span class="o">|</span> <span class="nb">attr_accessor</span> <span class="ss">:schedule</span>
<span class="o">|</span> <span class="k">def</span> <span class="nf">add</span><span class="p">(</span><span class="n">shift</span><span class="p">)</span>
<span class="o">|</span> <span class="vi">@schedule</span> <span class="o"><<</span> <span class="n">shift</span>
<span class="o">|</span> <span class="k">end</span>
<span class="o">|</span> <span class="k">end</span> <span class="c1">#=> nil</span>
<span class="o">>></span> <span class="no">Incredibles</span><span class="p">.</span><span class="nf">new</span><span class="p">.</span><span class="nf">instant_moving</span> <span class="c1">#=> "Shuuooo .......... you just moved!"</span></code></pre></figure>
<p>See, there's no method definition of super powers, but we can still use it because the inheritance in Ruby class. That's convenient!. But wait, they also got an intensive baby sitting schedule mentioned in my previous blog too:</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="o">>></span> <span class="no">Helen_shift</span><span class="p">,</span> <span class="no">Violet_shift</span><span class="p">,</span> <span class="no">Dash_shift</span><span class="p">,</span> <span class="no">Rob_shift</span> <span class="o">=</span> <span class="no">Array</span><span class="p">(</span><span class="mi">6</span><span class="o">..</span><span class="mi">15</span><span class="p">),</span> <span class="no">Array</span><span class="p">(</span><span class="mi">17</span><span class="o">..</span><span class="mi">21</span><span class="p">),</span> <span class="no">Array</span><span class="p">(</span><span class="mi">19</span><span class="o">..</span><span class="mi">23</span><span class="p">),</span> <span class="no">Array</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="mi">7</span><span class="p">)</span><span class="o">+</span><span class="no">Array</span><span class="p">(</span><span class="mi">22</span><span class="o">..</span><span class="mi">23</span><span class="p">)</span> <span class="c1">#This are their shifts in my previous blog.</span>
<span class="o">>></span> <span class="n">baby</span><span class="o">=</span><span class="no">Incredibles</span><span class="p">.</span><span class="nf">new</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">schedule</span><span class="o">=</span><span class="p">[]</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="no">Helen_shift</span><span class="p">)</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="no">Violet_shift</span><span class="p">)</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="no">Dash_shift</span><span class="p">)</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">add</span><span class="p">(</span><span class="no">Rob_shift</span><span class="p">)</span></code></pre></figure>
<p>What a busy schedule of the baby sitting! Don't you want to list all their shifts? Let's try it with each method. But wait, we didn't define the <code>each</code> method so Ruby will reject our request of using <code>baby.each</code>. Is there a way to list all of their shifts? It would be so nice if we can use all Enumerable methods here. In Carlson & Richardson's Ruby Cookbook Chapter 9.4, they showed a pretty <em><strong>mixin</strong></em> by adding the "include" and defining "each" to use all 22 Enumerable methods in one shot.</p>
<figure class="highlight"><pre><code class="language-ruby" data-lang="ruby"><span class="o">>></span> <span class="k">class</span> <span class="nc">Incredibles</span> <span class="o"><</span> <span class="no">Supermen</span>
<span class="o">|</span> <span class="kp">include</span> <span class="no">Enumerable</span>
<span class="o">|</span> <span class="k">def</span> <span class="nf">each</span>
<span class="o">|</span> <span class="vi">@schedule</span><span class="p">.</span><span class="nf">each</span> <span class="p">{</span><span class="o">|</span><span class="n">a</span><span class="o">|</span> <span class="n">a</span><span class="p">.</span><span class="nf">each</span><span class="p">{</span><span class="o">|</span><span class="n">b</span><span class="o">|</span> <span class="k">yield</span> <span class="n">b</span><span class="p">}}</span>
<span class="o">|</span> <span class="k">end</span> <span class="c1"># This each invokes block by calling the yield function.</span>
<span class="o">|</span> <span class="k">end</span> <span class="c1">#=> nil</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="mi">12</span><span class="p">)</span> <span class="c1">#=> true # noon shift is covered</span>
<span class="o">>></span> <span class="no">Array</span><span class="p">(</span><span class="mi">0</span><span class="o">..</span><span class="mi">23</span><span class="p">)</span> <span class="o">-</span> <span class="n">baby</span><span class="p">.</span><span class="nf">sort</span> <span class="c1">#=> 16 #4pm is not covered!</span>
<span class="o">>></span> <span class="n">baby</span><span class="p">.</span><span class="nf">include?</span><span class="p">(</span><span class="mi">16</span><span class="p">)</span> <span class="c1">#=> false #Indeed, 4pm's not covered.</span></code></pre></figure>
<p>Now we can marvellously use all core methods defined in the Enumerable module! Because there's no multiple inheritance in Ruby, in one customized class we can use methods from many other classes by <em><strong>mixin</strong></em>. Isn't that great?</p>
<p>By the <em><strong>inheritance</strong></em> the class got all methods defined in the father class, for example, Incredibles <em><strong>ARE</strong></em> supermen. By mixin we got specific methods from other class, and that's the idea of the <em><strong>composition</strong></em>, say, Incredibles schedules' <em><strong>HAVE</strong></em> Enumerable methods. There are many people hotly debating either inheritance or composition is better. In my point of view, as long as the handling of the repetitive codes is elegant, highly readable and maintainable, either or both ways can be applied in certain conditions.</p>