The Perfect Workflow, with Git, GitHub, and SSH
videos

The Perfect Workflow, with Git, GitHub, and SSH

In this lesson, we’ll focus on workflow. More specifically, we’ll use the helpful GitHub service hooks to automatically update a project on our personal server whenever we push updates to a GitHub repo.


Prefer a Video Tutorial?

Press the HD for a clearer picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Step 1 - Create a Git Repo

We certainly need some sort of project to play around with, right? Let’s do that right now. Using which ever tool you prefer (I’d recommend Structurer), create a new directory, called awesomeProject, and add an index.html file. Feel free to populate this with some gibberish markup for the time being.

With our test directory in place, let’s create our first Git commit.

If you’re unfamiliar with Git, I highly recommend that you first review “Easy Version Control with Git.”

Open the command line:

cd path/to/awesomeProject
git init
git add .
git commit -m 'First commit'

Those familiar with Git should feel right at home. We’re creating a Git repo, adding all the files to the staging area, and are then creating our first commit.


Step 2 - Uploading to GitHub

The next step is to upload our project to GitHub. That way, we can easily call a git pull to download this project from any computer/server we wish.

Again, if you’re not familiar with GitHub, and haven’t yet created an account, read Terminal, Git, and GitHub for the Rest of Us.

Begin by creating a new Git repository.

Create a Git Repo

Next, you’ll need to fill in some details about your project. That’s simple:

Details

And finally, since we’re already working with an existing Git repo, we only need to run:

git remote add origin git@github.com:Your-Username/awesomeProject.git
git push -u origin master
Git

With that out of the way, our awesomeProject is now available on GitHub. That was easy!


Step 3 - SSH

Now, we certainly need some sort of live preview for our project, ideally stored on our own server. But this can be a pain sometimes. Push your updates to GitHub, login to your server, manually transfer the updated directory, etc. Granted, this only takes a moment or so, but when you make multiple changes through out the day, this can quickly become a burden.

But one step at a time. We’ll tackle this dilemma in Step 4. For now, let’s simply pull in our Git repo to our server. To do so, we need to SSH in.

Depending upon your host, your SSH credentials will vary slightly. Search Google for “your-host-name SSH,” and you’ll surely find the necessary instructions. Once you’re ready, let’s move along:

We’ll use my personal server as an example:

ssh jeffrey-way.com@jeffrey-way.com
<enter your host password>

And with those two lines, we’re in!

SSH

Next, we cd to the parent directory of where we wish to store awesomeProject. For me, this will be: cd domains/demo.jeffrey-way.com/html/. Of course, modify this according to your own directory structure.

Git Clone

Let’s clone the GitHub repo now.

git clone git@github.com:Your-User-Name/awesomeProject.git

Give that command a few seconds, but, before you know it, that directory is now available on your server, and, in my case, could be viewed at: http://demo.jeffrey-way.com/awesomeProject.


Step 4 - Creating a Connection

The inherent problem at this point is that there’s no specific connection between our GitHub repo and the directory stored on our server — at least not an automated connection. For example, if we update our source files on our local machine, and then push the changes to GitHub:

git add index.html
git commit -m 'Added photo of dancing chicken'
git push origin master

These changes will certainly not be reflected on our server. Of course they won’t! In order to do so, we must – once again – SSH into our server, cd to the awesomeProject directory, and perform another git pull to bring in the updated source files.

Wouldn’t it be great if, every time we pushed updates to GitHub, those new source files were automatically updated on our live preview server?

As it turns out, we can do this quite easily with GitHub service hooks.

GitHub Service Hooks

You can access this page by pressing the “Admin” button from within your GitHub repo, and then clicking “Service Hooks.” The “Post-Receive URL” option will instruct GitHub to send a POST request to the specified page every time you push to your GitHub repo. This is exactly what we need!

“We’ll hit these URLs with POST requests when you push to us, passing along information about the push.”

To make this work, we’ll need to create one more file that will handle the process of performing the git pull. Add a new file, called github.php (or anything you wish – preferably more vague), and add:

<?php `git pull`; 

So now you’re thinking: “Jeff’s gone crazy. You can’t put a Bash script into a PHP string.” Well…yes you can, once you realize that those aren’t single quotes above, they’re back-ticks.

When you wrap a sequence in back-ticks, in PHP, it’ll be treated as a Bash script. In fact, it’s identical to using the bash_exec function.

Save that file, and upload it to the awesomeProject directory on your server. When finished, copy the url to that file, and paste it into the “Post-Receive URL” textbox. In my case, the url would be http://demo.jeffrey-way.com/awesomeProject/github.php.

With this in place, every single time you push to your GitHub repo, that file will be called, and the awesomeProject directory on your server will auto-update, without you needing to move a finger. Pretty nifty, ay?


You Also Might Enjoy:

Note: Want to add some source code? Type <pre><code> before it and </code></pre> after it. Find out more
  • http://www.biblestudyinteractive.com Shaun

    I can ssh into my domain, but once I run any git commands it says “command not found.” Please excuse me if I missed something obvious. This whole command line stuff is totally new to me.

    • Shaun

      To be more specific, the problem happens when I type in “git clone …” which it then responds with “command not found.”

      • Ian

        It sounds like you don’t have git properly installed. Run a git –version and/or git help. If those still return command not found, then you most likely don’t have git installed properly. Look into how to install git onto your server.

  • Irfan Zaheer

    Thanks for the great post, really wonderful explanation.
    But just having some difficulties with command git pull.
    I mean when i manually execute this command on SSH it asks for pass-phrase and after verifying its update the code.
    How can i avoid this pass-phrase entering step, & may be this is the primary reason that when i do push from my local computer, it does updates on github but unable to do it on my own server?

    Please if any body come across same problem, please help!

    Thanks.

    • Todd

      Irfan,
      In order to use ssh without password you will need to generate ssh keys. This will typically be the same process on most servers/hosts, but will vary depending on whether you are using osx/linux/windows. This might be of assistance: http://oreilly.com/pub/h/66

  • Met

    Very nice workflow,
    but how do you manage those files where local version differs from online (eg: db settings)?

    Great tut, anyway (as always).
    Met

    • Met

      Ignore, maybe?

  • Aron

    Very nice tutorial.

    We made a similar video demo (or tutorial if you like) for our internal team about the Integrator Manager workflow. Our developers have been using SVN for long years before converting to Git, and it was a major conceptual change that required multiple tutorial sessions like this. You may also find it worthwhile: “Git Workflows Demo: What is the Integrator Workflow?” http://www.youtube.com/watch?v=67zQHNO2gaQ

  • Niels van Aken

    Won’t the Github passphrase for your ssh key be a problem?

    • Niels van Aken

      Sorry, I did not realize there was a second page of comments discussing the passphrase.

  • Devin Dombrowski

    I feel obligated to share my story involving this type of workflow.

    I spent hours once trying to set up a github webhook post but no matter what I did it simply would not work. I was loosing my mind. So I went for a walk and tried to clear my thoughts. Then it came to me. That “ah ha!” moment. Turns out the the reason the posts were being blocked was actually a “nice” thing. I had some CSRF blocking going on. My site thought I was trying to hack it and was blocking all my request.

    So, morally of the story…. Go for a walk every now and then.

    And also, if your having trouble with github webhook posts, dont forget about CSRF if you are doing that.

    • Devin Dombrowski

      Ok follow up…

      If we remove CSRF screening on the page that github is posting to, then is that bad? Aren’t we becoming vulnerable? Yes, technically we are. So how can we manage?

      Here are a few ways.
      1. Is the POST coming from a github public IP? (ie: 207.97.227.253, 50.57.128.197, 108.171.174.178)
      2. Add a secret url parameter to the url to post to. (ie: domain.com/github.php?key=super-secret) Then in github.php check if $_GET['key'] === ‘super-secret’
      3. And lastly, just because, make sure isset($_POST['payload']) is true

      If any of these are false then die();

  • http://www.tegdesign.com Tegan Snyder

    If anybody has troubles setting up the WebHook and PHP pull process. I finally got it to work after fiddling around for sometime and using a bash script. I detail the process here: http://www.tegdesign.com/2012/git-webhook-php-post-receive-pull-method/

  • http://mac-blog.org.ua Alexandr Marchenko

    What about conflicts? If I commit something that can not be merged, as I understand git pull will modify files with conflict information and they will no longer be working.

  • Derick

    Hi guys first I want to thank you for making this video, it’s the first git tutorial I’ve been able to “git” my head around so to speak. I’ve been able to do everything in this tutorial except one thing and that’s the automatic pull from github each time there’s a push. I think I know what the problem is but if someone could confirm it or explain what I would need to do to get it to work would really be appreciated.

    I have the github.php file in the public folder e.g. example.com/github.php I’ve added that url to the webhook in git but the actual repository is one level lower then the public folder, that’s where the majority of the files I work with are. So I’m guessing that the webhook receives the new push it hits the url but but the script in github.php is trying to pull from that same directory … is that right?

  • Netto

    I follow exactly as you said, but my file in web-server don’t changed. Look at my git push:

    $ git push
    Counting objects: 5, done.
    Delta compression using up to 4 threads.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (3/3), 313 bytes, done.
    Total 3 (delta 1), reused 0 (delta 0)
    remote: bb/acl: is allowed. accepted payload.
    To git@bitbucket.org:/.git
    c335e64..b428bef master -> master

    The only thing that can’t change is in webserver. Could you help me?

  • http://www.ajency.in Suraj

    Coool. Thanks…

  • Billy

    Awesome read. Just curious, but couldn’t anyone push something to github.php if they knew it was there? Am I missing something and/or being a worry wart?

    Thanks!

    • http://imkreative.com Clemente Gomez

      You usually have to give people permission to push stuff to you’re repository.

  • http://www.horesh-studios.com Motti Horesh

    Hey Jeffrey, Great post!

    How would you go around setting multiple development tiers using that route? for example, dev-test-prod automation?

  • http://imkreative.com Clemente Gomez

    That is such a time saver. Never thought to do it like that anyway. Good Tut as Always!

  • Carlton

    Is there also a way to do this if I am running .Net asp sites on IIS?

  • Calvin

    Hi,

    I’m asked for a passphrase when I use Git Pull. Is it wise to automate the input of this in github.php and how would I go about doing it?

    • http://twitter.com/9eteyGreen Pete Lower

      I’m wondering the same thing here…

    • http://dev.xscheme.de/ Yannick

      I think you might be using SSH (with a passphrase-protected private key) to pull your repository. You should not automate this since it requires your passphrase to be known to the PHP script (and thus to anyone getting access to it) which IMHO is not the wisest thing to do.

      But if you initially clone your repository using either the “HTTP” or the “Git Read-Only” links on your project’s GitHub page, you should be on the safe side whilst not having to worry about passphrases at all. I’d prefer the “Git Read-Only” option since read-only access really is all you need to accomplish what was outlined in this article.

      Cheers.

  • james

    If I am trying to use this trick with a Drupal site, where would I put the github.php file?

    • james

      I followed the steps to the t, can’t get it to work. Please help

      • http://gabrielmanricks.com/ Gabriel

        what host are you using?

      • James

        I’m using justhost.

      • http://gabrielmanricks.com/ Gabriel

        JustHost doesn’t seem to support git, but they do have ssh. when you login through an ssh terminal do they have git installed?. If they do then you should make sure the github.php file is in the root of your repository. and the last thing to check is where on the server you cloned the repo to. If for instance your websites root is a folder called “public_html”and you clone the repo in there, then your website will not show up because it is in a nested folder on your site. for example if your repo’s name is “drupalProject” and you clone it in your sites root then to access it you would have to go to “yourSiteadress/drupalPorject/”. And remember the first git clone has to be done manually through SSH.
        As to the above problem with the nested folder this can be fixed with a .htaccess file and either mod_rewrite or mod_alias with something like; “AliasMatch ^/(.*)$ /drupalProject/$1″
        And if your host doesn’t have git it might be possible to install it through ssh. I hope this helps if you have any further questions feel free to reply.

      • james

        Hi Gabriel. Thanks for the detailed explanation. Justhost uses cpanel. I’m using one domain (in the www folder of cpanel) to do all my test sites. I have a Drupal installation (my test site for this exercise) in a nested folder of that domain. I placed the github.php file in the root of that test site folder. Do I need to use your htaccess example to make this work? Thanks, again.

        James

      • http://gabrielmanricks.com/ Gabriel

        Do you have SSH access

      • James

        Yes, I do.

      • james

        @gmanricks:disqus : Yes, I do

      • http://gabrielmanricks.com/ Gabriel

        And does the git command work when you log in through ssh

      • james

        @gmanricks:disqus : Yes, it is working, now. I had a few problems with it yesterday. I am now able to use the git pull and git push commands.

      • http://gabrielmanricks.com/ Gabriel

        Now the php script should work ( it has to be placed in the folder that was cloned from github) and from what you mentioned before it seems like you will need the .htaccess file if you want your site to open into it.

  • bobzyouruncle

    Great post thanks. I dont understand why tutorials explaining git, github dont use flow diagrams displaying pulls clones etc. to visually help noobs understand whats going on…. or perhaps they are out there and I havent seen any…?

  • Rob Bennet

    Very awesome idea, kudos!

  • http://www.step4wd.com/ M. Ahmad Zafar

    The remote repository on your own server is a bare repository? It would have a .git folder?

    • http://twitter.com/joshtronic Josh Sherman

      Structure your code so that .git is above the public doc root, git pull can be executed from any directory in the tree

      • http://www.step4wd.com/ M. Ahmad Zafar

        Right!

    • http://twitter.com/joshtronic Josh Sherman

      OH, could also rock out .htaccess (or the equivalent for the web server you prefer) to block access as well

  • http://twitter.com/joshtronic Josh Sherman

    Love it, nice and easy… just to add to that, I’d wrap the PHP file with a sanity check for a “secret word” just to add a bit of security. Yeah, you can call me paranoid ;)

    • http://twitter.com/joshtronic Josh Sherman

      oh even better, didn’t realize Github tells you the public IP’s they are sending from… definitely would check for those :)

  • Anders

    Thanks a lot – just a minor roadblock, which I cannot seem to ‘break’: Any time I run the php script, I get this is the Apache error log: error: cannot open .git/FETCH_HEAD: Permission denied

    I have chmod’ed FETCH_HEAD to g-w, but still no luck. Any ideas?

  • shaan

    m just curious correct me if i am wrong what if there is a load balancer in between and below load balancer there are 3 servers serving site ??

  • http://twitter.com/heyallanmoreno Allan Moreno

    Does this tutorial work using the GitHub.app? I don’t really see my self using terminal :P

  • http://twitter.com/wunderdojo wunderdojo

    I’ve been trying to figure out a good way to do this to handle updating a handful of sites that use a custom software package I wrote. Your directions were perfect, now have all of them updating automatically whenever I push to the production branch. Thanks a ton.

  • Carsten

    wooooza this made my day!
    I still cannot post from my ipad then but hey, automatic push with versioning system. great.

    thanks for writing this down

  • http://twitter.com/AshConnell Ash Connell

    Alternatively, If you don’t want it to update the live server on every commit, but still want a quick way for the live server to pull, just add a link to the bash script to your bookmarks bar… tadah! insta-pull!

  • skube

    This is great – I wish I could get it to work though :(

    For some reason, on my server when I attempt a
    git pull
    I get the following returned:
    git: `pull` is not a git command.

    As a work around, I tried doing a fetch, followed by a merge:
    git fetch
    git merge origin/master

    This “manual” method does work. However, when I then attempt to further automate by putting it into my github.php webhook, nothing happens.

    • skube

      update: apparently, “git-core wasn’t mounted under my jailshell file system and this was causing the issue.”

      Unfortunately, I still cannot automate, since I pull still prompts for my passphrase.

      • skube

        Update: cloning the github repo via http rather than ssh seems to stop the passphrase prompt

    • skube

      another update: i have everything almost working except for the last bit. As I test, I tried hitting the webhook url directly – nothing happened. However, when I use the terminal and try:

      php github.php

      Then the `git pull` executes, albeit with a possibly unrelated warning:
      Failed loading /usr/local/ioncube/ioncube_loader_lin_5.3.so: /usr/local/ioncube/ioncube_loader_lin_5.3.so: cannot open shared object file: No such file or directory

      From the comments below it appears “Apache user can’t exec a bash script, but the terminal user can.”

  • http://www.facebook.com/ylinhtun Ye Lin Htun

    It is not working in Amazon EC2 server, isn’t it? I haven’t received any error.

  • http://www.facebook.com/slapback Jamie Davison

    Pretty nifty but I can’t think of anything more dangerous in a production environment

  • Marcio

    It is not working in Amazon EC2 server, isn’t it?