I came across this nice online beginner’s course on Elm here. Still can’t believe that it’s free to learn, I’m just watching the videos and expecting a paywall to come up anytime, but so far, 3 lessons in, it’s been great. 🙂
I finished building my webscraper yesterday. I wanted to learn how to build one, and I followed some online tutorials and used the nokogiri gem. It was quite a smooth and interesting experience… Nokogiri is useful, you can choose to target the elements you want to scrape using CSS or Xpath.
I’m not sure what’s Xpath, okay a quick google search says it’s syntax for targeting parts of an xml document. XML is similar to HTML, except the syntax is more descriptive, like actual words in the tags instead of <p>s, <br>s and what not.
Okay, so anyway, while learning to build the webscraper with nokogiri, I learnt some cool new things.
- I learnt about other webscraping gems like spidey and other tools; one useful tool is a Chrome extension called SelectorGadget (the name sounds similar to Inspector Gadget, that 80s cartoon series), it helps you easily find and group the CSS/Xpath elements you want to target.
- I also learnt to create a random User-Agent for when Nokogiri targets a html page, because if you don’t specify a User-Agent name, you get a default one, and after a few tries, you will get a 429 error code for sending too many requests to the site/page you want to scrape.
- Lastly, no matter how small the project is, there’s always something new to be learnt. I guess this is because of how new I am to web dev/programming.
My initial goal to scrape my own diaryland website is accomplished. Good job lol. I followed an online tutorial that was very informative. This one.
What next? I think there’s improvements to be made:
- For one, I’ll need to learn how to create a background service job for scraping. So that the rails server can do other things while it’s scraping; like serve the initial results first.
- And pagination, I think that’s easier with the Kaminari gem.
Okay, this is it. Oh yeah, another thing I learnt, well I learnt that i have about 680++ diaryland entries. That’s amazing haha. I’ve been writing stuff at my diaryland page since I was 24? That’s in 2004, so it’s been more than 13 years. Wow. I’m surprised diaryland is still around frankly.
So okay. See you. Yeah in case you want to scrape your own diaryland page, with rails, if you have either, you can see the source code here.
I made a simple Facebook chatbot demo. You can message it at its Facebook page and your email and message will be saved to this website. To check out the source code, just go to my github.
I decided to build this because I think it might be easier to demo the link between Facebook messenger to rails server to Api.ai. If you would like to see more of what Api.ai can do, maybe you can check out Rem, the mood-journaling chatbot I did with my project mates in Alpha Camp. There’s more conversation design there.
I used facebook-messenger gem and api-ai gem to link the backend with both facebook and api.ai. I think tinkerbox has a message-quickly gem that allows your rails app to integrate with Facebook messenger too for chatbot functions, it looks quite nifty, i’ll try it out some time. Maybe after I try building a webscraper with rails (following an online tutorial). I’ve always wanted to build a webscraper, it’ll help me extract and save my diaryland postings. 🙂
Tonight was the night of our Alpha Camp project presentation. It marks the end of our 12 week web development course and we will soon be embarking on the next chapter of our life – back to work and reality! (More coding and learning.)
So, me, Nathan and Evan, we worked on a chatbot called Rem for our capstone project. Rem is supposed to provide a listening ear for users and comfort them when they feel depressed. Besides this, Rem also incorporates techniques from CBT (cognitive behavioural therapy) – the two techniques are Mood Journaling and Gratitude Journaling. Users will be encouraged by Rem to record down their moods throughout the day, and also record down what/who they are thankful for. These techniques have been proven to work by the way.
So yeah, we bought a domain name and are running a hobby server at heroku for this. We’ll see how it goes. You can check it out at https://remchat.herokuapp.com
Just sign up for an account, check your email for a confirmation link. And then you can enter our site to start chatting.
Also, after you have signed up, you can also go to facebook messenger https://www.facebook.com/remjournal and find Rem to chat. It will prompt you to link your fb account to the heyrem site, so just do that and you’ll be fine.
The digital marketing girls (Hui Sze and Desiree) have made a nice digital marketing landing page for Rem at http://remmoodjournal.strikingly.com/ You can check it out for more details regarding Rem.
For Rem, we used ruby on rails to communicate with FB’s webhook for the messenger integration. And for the AI language processing, we used API.ai from Google. It’s a free service at the moment. I think it really helped that we approached the project together and supplied each other’s knowledge with our own learning at the same time. So that a daunting (and unknown) task eventually became more manageable.
A single page app, random dad jokes.
An online bookstore website done with project mate Evan; includes Braintree online payment system, Facebook login via Oauth, carrierwave & AWS image storage. Try out the site here.
Users can sign up using their Facebook accounts to buy books. (But please note, we haven’t submitted it for review with Facebook, so you can’t actually sign in using your FB account unless you’re the site’s developer like us, so you’ll have to sign in using your email.) When the customer buys a book, the stock quantity will decrease. And when the customer successfully pays for the book, he/she will receive an email notification.
If you are an admin, you could also sign in to upload books, a book image, descriptions, and stock quantity. Admins can also view orders and change order statuses, for example, ‘shipped’ status.
The layout (navbar) is similar to twitter 2.0, because we used bootstrap and this navbar again. This site could be improved with a better grid layout, and functions like a book title and author search.
A twitter clone website done during Alpha Camp coding bootcamp with my project mate, Ash. Visit it here
Signed-in users can follow other users and see their feeds populated by those they follow. Has a search function to search user names and posts.
Just a simple site, could do with more code refactoring so that there’s less requests on the database, and better rspec tests coverage. Images are also missing because I didn’t use AWS to host the user-uploaded images, the images are on Heroku and Heroku stops and reruns its dynos from time to time, so the images are gone now.
You have to register for an account, sign-in, and follow other users to see their posts.
This website mainly demonstrates our knowledge of ORM (object-relational-mapping), and model associations/relationships.