The Perfect Workflow, with Git, GitHub, and SSH
videos

The Perfect Workflow, with Git, GitHub, and SSH

on Apr 18th 2011 with 136 comments

In this lesson, we’ll focus on workflow. More specifically, we’ll use the helpful GitHub service hooks to automatically update a project on our personal server whenever we push updates to a GitHub repo.

Prefer a Video Tutorial?

Press the HD for a clearer picture.
Subscribe to our YouTube and Blip.tv channels to watch more screencasts.

Step 1 - Create a Git Repo

We certainly need some sort of project to play around with, right? Let’s do that right now. Using which ever tool you prefer (I’d recommend Structurer), create a new directory, called awesomeProject, and add an index.html file. Feel free to populate this with some gibberish markup for the time being.

With our test directory in place, let’s create our first Git commit.

If you’re unfamiliar with Git, I highly recommend that you first review “Easy Version Control with Git.”

Open the command line:

cd path/to/awesomeProject
git init
git add .
git commit -m 'First commit'

Those familiar with Git should feel right at home. We’re creating a Git repo, adding all the files to the staging area, and are then creating our first commit.

Step 2 - Uploading to GitHub

The next step is to upload our project to GitHub. That way, we can easily call a git pull to download this project from any computer/server we wish.

Again, if you’re not familiar with GitHub, and haven’t yet created an account, read Terminal, Git, and GitHub for the Rest of Us.

Begin by creating a new Git repository.

Create a Git Repo

Next, you’ll need to fill in some details about your project. That’s simple:

Details

And finally, since we’re already working with an existing Git repo, we only need to run:

git remote add origin git@github.com:Your-Username/awesomeProject.git
git push -u origin master
Git

With that out of the way, our awesomeProject is now available on GitHub. That was easy!

Step 3 - SSH

Now, we certainly need some sort of live preview for our project, ideally stored on our own server. But this can be a pain sometimes. Push your updates to GitHub, login to your server, manually transfer the updated directory, etc. Granted, this only takes a moment or so, but when you make multiple changes through out the day, this can quickly become a burden.

But one step at a time. We’ll tackle this dilemma in Step 4. For now, let’s simply pull in our Git repo to our server. To do so, we need to SSH in.

Depending upon your host, your SSH credentials will vary slightly. Search Google for “your-host-name SSH,” and you’ll surely find the necessary instructions. Once you’re ready, let’s move along:

We’ll use my personal server as an example:

ssh jeffrey-way.com@jeffrey-way.com
<enter your host password>

And with those two lines, we’re in!

SSH

Next, we cd to the parent directory of where we wish to store awesomeProject. For me, this will be: cd domains/demo.jeffrey-way.com/html/. Of course, modify this according to your own directory structure.

Git Clone

Let’s clone the GitHub repo now.

git clone git@github.com:Your-User-Name/awesomeProject.git

Give that command a few seconds, but, before you know it, that directory is now available on your server, and, in my case, could be viewed at: http://demo.jeffrey-way.com/awesomeProject.

Step 4 - Creating a Connection

The inherent problem at this point is that there’s no specific connection between our GitHub repo and the directory stored on our server — at least not an automated connection. For example, if we update our source files on our local machine, and then push the changes to GitHub:

git add index.html
git commit -m 'Added photo of dancing chicken'
git push origin master

These changes will certainly not be reflected on our server. Of course they won’t! In order to do so, we must – once again – SSH into our server, cd to the awesomeProject directory, and perform another git pull to bring in the updated source files.

Wouldn’t it be great if, every time we pushed updates to GitHub, those new source files were automatically updated on our live preview server?

As it turns out, we can do this quite easily with GitHub service hooks.

GitHub Service Hooks

You can access this page by pressing the “Admin” button from within your GitHub repo, and then clicking “Service Hooks.” The “Post-Receive URL” option will instruct GitHub to send a POST request to the specified page every time you push to your GitHub repo. This is exactly what we need!

“We’ll hit these URLs with POST requests when you push to us, passing along information about the push.”

To make this work, we’ll need to create one more file that will handle the process of performing the git pull. Add a new file, called github.php (or anything you wish – preferably more vague), and add:

<?php `git pull`;

So now you’re thinking: “Jeff’s gone crazy. You can’t put a Bash script into a PHP string.” Well…yes you can, once you realize that those aren’t single quotes above, they’re back-ticks.

When you wrap a sequence in back-ticks, in PHP, it’ll be treated as a Bash script. In fact, it’s identical to using the bash_exec function.

Save that file, and upload it to the awesomeProject directory on your server. When finished, copy the url to that file, and paste it into the “Post-Receive URL” textbox. In my case, the url would be http://demo.jeffrey-way.com/awesomeProject/github.php.

With this in place, every single time you push to your GitHub repo, that file will be called, and the awesomeProject directory on your server will auto-update, without you needing to move a finger. Pretty nifty, ay?

You Also Might Enjoy:

 
Enjoyed this Post?

Subscribe to our RSS Feed, Follow us on Twitter or simply recommend us to friends and colleagues!

By

Rollover to read this author's bio or click through to see a full list of posts by this author.

I'm the editor of Nettuts+, and love making the process of building for the web as easy as possible. Also, I'm finishing up my latest book on Laravel Testing! If you want be friends, say hi on Twitter.  

Plus Premium
 
 
Note: Want to add some source code? Type <pre><code> before it and </code></pre> after it. Find out more
Newer Comments →
  • Rohith

    This was what i was exactly looking for.
    Had a little struggle with setting ssh keys, but it is working perfectly.

    Thanks for such a awesome post.

    • Rob

      I’m having trouble with my SSH keys through Dreamhost. I’ve followed all tutorials and debugs on github site as well as an exhaustive Google and StackOverflow. Anyone else have similar issues and find a fix?

  • http://creolab.hr Boris Strahija

    Very nice. Is something like this possible on BitBucket?

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Sorry – not sure.

    • cahva

      If you’re referring to the post-receive url then yes. You can find that in Bitbucket in your repo menu under Admin->Services

    • http://www.cyberstream.us Eli Mitchell

      Yes it is possible. I fought through it for a couple days, but finally figured it out. Here’s how I implemented Mercurial/BitBucket and SSH with Netbeans IDE. The post will probably still help even if you aren’t using Netbeans.

  • Rizky Syazuli

    but at the start, you still have to pull your repo manually to your server right? and that’s also assuming that hosting provider have git installed.

    • http://scottlee.me Scott Lee

      Yup.
      I’d really be interested if there was a way to accomplish this without having Git installed on the remote server.

    • Josh

      If you’re with mediatemple, they have git installed by default on all their servers. I’m sure you have seen their hosting deals before, I would definitely recommend it.

    • http://paulirish.com Paul Irish

      Curiously enough, github has SVN read-only support..

      https://github.com/blog/626-announcing-svn-support

      So you can still pull from a github repo if you only have SVN on the server.

  • choise

    the service hook trick is awesome

  • http://1timetracking.com Derek Organ

    Minor detail but in your video you are doing a ‘git add index.html’ every time you change the file. You only need to do this for files that are not already tracked by git.

    Like the way you use terminal in full screen like that must google it.

    lastly I wondering is there any potential security flaws having a github.php file that runs on the command line call. I can’t think of any but its always a dangerous thing. also what user is doing the command e.g. is it www-data and can that lead to issues in git if I’m usually setting up the git with my own user e.g. ‘jefferyway’?

  • ElCynico

    Thanks for the service hook tip, very nice!

    However, I have set up my Git SSH key with a passphrase and every time I do a ‘git pull’ it asks for a passphrase to the private keyfile. Apparently GitHub service hook doesn’t carry this passphrase in the POST request and thus fails.

    Any possibility to fix this?

    • http://www.rockhopperdigital.com Benjamin Zalasky

      ElCynico, you need to use ssh-agent (or System Keychain if you’re using OSX Snow Leopard). You can find out all about it here: http://help.github.com/working-with-key-passphrases/

    • http://paulirish.com Paul Irish

      For the server git pull, i would recommend to pull from the HTTP URL. In the repo view, you can change the URL to the ‘http’ view instead of ‘ssh’ and you wont need credentials (or any of the passkey setup).. The only downside is you can’t push from your server (which is probably okay).

    • tom

      I think you could just use ssh-agent.

  • Will

    On my server every time I run a GIT command it asks me to enter in my passphrase. Does anyone know if there is a way to disable this?

    • Will

      My friend helped me. Turns out I just needed to recreate my public ssh key and leave the passphrase blank. Either that or in my shell script use except (http://en.wikipedia.org/wiki/Expect)

  • http://linein.org/blog/ John Veldboom

    How would you do this same thing but have certain branches push to different servers? For example have all the pushes into “master” go to the production server while everything else go to the development server.

    Thanks!

    • Burak Erdem

      I wonder exactly the same. I wish someone here answered that question.

  • http://www.jchilders.com James

    Fullscreen terminal: Visor http://visor.binaryage.com/

    For SVN people: you can accomplish ALL of this using Beanstalk http://www.beanstalkapp.com

  • Harm de Wit

    Very nice video. I will definitly try this in my next projects.

    I don’t know what you use to get the terminal fullscreen, but it inspired me to set the the terminal in a keyboard maestro application shortcut and in a full screen opacity to get the same effect :).

  • http://jaredeasterday.com Jiert

    Jeffery, you’re a god among men. I’ve been looking for something like this for a long time. I recently began messing around with the Cloud 9 IDE, and wasn’t sure how I would push my changes to my server (it doesn’t support FTP yet).

    Not only does this solve my Cloud 9 question, but it also removes steps when working locally.

    thank you thank you thank you.

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Trust me — the Gods are *way* better than me. :) But yeah – SSH knowledge is a good tool to have.

  • http://itcutives.com Jatin

    I am a big fan of SSH and GitHub. Really nice tutorial. Good Work Jeffrey. Looking forward to your other tutorials.

    Thanks.

  • Kris Reynolds

    Sorry, i watched the video and still dont understand the benefits of github, when i could just as easily work with my localhost make a quick backup and then upload the changes via ftp with filezilla…

    Is there something im missing, is this really beneficial?

    • http://envexlabs.com Matt Vickers

      You won’t be able to track changes between each backup without downloading the backup, unzipping everything and then using a program to compare the files.

      Plus, writing 3 lines in terminal > downloading, zipping and uploading a bunch of files.

  • http://www.szkolenia24h.pl barat

    Nice trick, but why I can’t file such script only if master/origin branch is updated…
    Ofcourse I can use PHP to get $_POST data and decide what to do with such commit, but why fire script, if proper branch isn’t updated? ;)
    But I thing that this is more suggestion for github staff, not for You ;)

  • Daniel

    Great tutorial Jeffrey =)

  • http://newarts.at Drazen Mokic

    Very nice, i definitely want more real-world workflow tuts

  • http://www.matt-bridges.com/ Matt Bridges

    Excellent as always, Jeffrey, but I think there is an even easier way to do this, especially if you are on a closed project, such as for a client.

    Springloops.com has been around for a while using SVN, but with their new 2.0 release (which I am currently using on a few projects) Git is supported. You can easily setup multiple servers, have collaboration, and have Springloops automatically push files to the server without having to create the extra ‘github.php’ POST file on your own server. It rocks!

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Interesting – checking it out now.

  • Yorick

    Uhm.. it seems that using github is kinda useless if you have a server wich has git on it.. with some more configuration you could set up Gitosis or some other git wrapper (http://wiki.dreamhost.com/Gitosis).

    For me this is the perfect solution, (no 3rd party), although github has it’s advantages..

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Maybe – but for those like me who use GitHub on a regular basis, it makes sense.

  • http://www.lk-webdesigns.co.uk LKenneth

    Excellent stuff, thanks for this.

  • http://www.gorelative.com Mike DeVita

    So question… how secure is this? What prevent’s a person from just visiting github.php via URL?

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Right – this would probably be used for much smaller projects, like demos and such. And then, you should also make the github.php file a unique name so that it’s not easily accessed.

    • mrgstiffler

      You could prevent the pull from running unless the file is being accessed from GitHub.

      • ElCynico

        Restricting should be easy in Apache .htaccess. Maybe you could limit file access to POST requests only.

        Something like this?

        Order allow,deny
        deny from all

        Also, unique filename for the script helps a lot, though it has to be updated manually every now and then to GitHub.

    • http://www.nouveller.com/ Benjamin Reid

      You could always pass a key to the page, example.com/git.php?key=your_salted_string and just kill the script if the key is not correct.

  • http://bensheedy.com Benjo

    I haven’t read this yet. But judging by the title, this is the article I’ve been waiting on.

    Thank you jeffrey. You rock my world.

  • http://bensheedy.com Benjo

    Is it be possible to push to a personal server without the GitHub account?

    This is the closest info I’ve found? http://toroid.org/ams/git-website-howto

    (Trying to figure out security issues and remove the reliance on the third party account if possible)

  • http://www.modernooze.com sam – web design dorset

    Thanks interesting artcle, i shall give it a go

  • DJ

    Backtick – A.K.A. the “grave” for those of us who learned english in the era of 8-track tapes and record albums.

  • http://vdwave.com itsmrwave

    The timing of this tut is really … CREEPY! :-) Just been changing my workflow to git this week.

    I think you should really include in the tutorial that your web-host must have git installed. I tried to do this, however to push changes to my web-host directly instead of passing to github but my web-host doesn’t have git installed! *sigh*

    There is a way to install git if your web-host doesn’t have git installed. Download git source files upload to you home directory and ‘make’ then ‘make-install’ ALL in your home directory by specifying and optional install directory using –opt-directory=”blah-blah-blah”.

    BUT, the flaw is that gcc must be installed to to able to compile the source in the first place :-) Sorry.

    For those concerned about security, you can skip the service hook. Add your web-host directory as another remote location but that will need you to push to it whenever you want the changes to be reflected. If like Jeffrey you use github also. Then that would need you to push to github THEN your webhost.

    For those who don’t understand why Jeffrey finds this method useful … its because, with this method you just need to push ONCE to github and your site just pulls from github. Two birds with one stone kind of methodology. And also simply because github ROCKS. Somethings just don’t have a reasonable explanation to be honest.

  • http://www.romainboichat.ch Romain Boichat

    Hi,

    Thank you for this nice & simple introduction. I have been wanting to use an easy version control system.

  • DJ

    Jeff… sorry, but I don’t see anywhere else to communicate with Nettuts on a general, non-article topic except for comments. I just want to thank Nettuts+ for NOT participating in the BAIT AND SWITCH scam today over premium discount coupons. [If you did initially, you have at least taken down the post after the apparently ONE coupon was used.] Over the years, many of us (well at least I) have begun to trust your recommendations and comments. It’s nice to know that, even though earning and maintaining trust doesn’t seem to apply to other Envato editors, your integrity is still intact. [Besides, with over 81,000 registered followers and countless thousands un-registered who needs a sham?] Thanks… and I’m sure you’ll be glad not to have to endure the firestorm of negative comments that is occurring over on the other sites.

    And, oh, no pressure, just wondering when the “cheerful website” post you promised for 80,000 followers might be posted?

    • http://www.jeffrey-way.com Jeffrey Way
      Author

      Hey DJ –

      Thanks, but it certainly wasn’t a scam — more a clerical error that has since been fixed. :) Envato is about as honest company as you can get.

      • dj

        Jeff… Yes, I agree with your assesment – at least that’s the image that I have gotten from about every thing I’ve seen over the past three years. I’ve watched how you personally handled issues and even how Nettuts+ (and Envato) handled all the furor in the comments back when promotion of premium content began being promoted on “free” pages and readers were upset at titles not clearly reflecting the “sales” nature of the post. You openly took care of it and titles now clearly don’t set any unclear expectations.

        The high ethical expectations we have of Envato is what, I’m sure, is fueling all the “unrest” in the comments on other Envato sites; otherwise, we would merely erase the blogs from our bookmarks and move on. Obviously we don’t want to; cause, really, where would we go?

  • http://www.nathan-long.com Nathan Long

    If you don’t have ssh access on the server (read: cheaper server space), you might need to install René Moser’s git-ftp (https://github.com/resmo/git-ftp) to use git to upload files. You won’t be able to use github’s service hooks this way though :( Requires two pushes.

    Unless I’m wrong – in which case I’d love to be corrected.

  • http://www.hirejosh.com Josh

    Awesomesauce. I’ve been meaning to find a way to “auto pull” to my live server when pushing from local. You rock.

  • http://tommybrunn.com Tommy Brunn

    Neat trick with the service hook. I’ve never seen that before. It’s a shame most cheap hosts don’t have git installed.

  • http://www.thegoodlab.com Bryant

    I would highly recommend also integrating Capistrano into this setup. Git + Guthub + Capistrano lets you deploy with 2 or 3 commands. They also have a gem for a “rails-less” deploy, for non rails projects, which I use for nearly all mine.

  • jn

    Argggh why this not working!!

    i can pull from my project directory on the server, but when i try to run the php bash script nothing happens :-/

  • http://www.giocc.com Gio Cielo

    We have a similar setup at MetaZaku that Yorick described above using Git, Gitolite, Redmine, and multiple servers. Instead of pushing updates to a URL, we configured the repository to SSH into remote servers with a ‘git-pull’ SSH trigger. Similar but not the same.

    The biggest difference between our setup and this system is that we use Redmine to manage the issues, wiki, and code browsing. It’s like a personal GitHub for your team or company.

  • http://webdecs.wordpress.com Denis

    Excellent tutorial man!

  • Martin

    @jeffrey, thanks for the article, its great. Got everything running except for the auto pull using the backtick command.

    Pretty sure its something I need to configure back at mediatemple gs server… but not sure what or how.

    I create a .profile file for using an ssh-agent but still if i do the git pull manually thru shell its still asking me for passphrase so I assume the github.php file runs into the same issue and does not do the pull automagically.

    thanks for any insight :)

  • http://www.marmaladeontoast.co.uk Terry

    Really nice idea but perhaps I’m being a doofus here. If you’re triggering the git pull from a PHP file served by Apache, will the git pull not be run as the Apache user (www-data or like)? In which case does that user need SSH keys etc.?

    Basically, if I hit github.php in a browser, I get nothing. If I run the github.php from the command like, “php github.php”, it works…

    It might be because I’ve trying to do this against a private repo that I’m having so many problems with it.

    Any ideas?

    • http://flyingcloudmedia.com Casey Becking

      I’m having the same issues. Apache user cant exec a bash script but the terminal user can. Can anyone explain a way around this issue?

    • Roger

      Ditto here. Trying to figure out how to:
      - Enable the proper permissions on the local repo for the Apache user on Mac server
      - Setup SSH key for the Apache user to successfully git pull from a private repository

      • Roger

        Finally figured it out (perhaps in somewhat insecure fashion).

        - Had to eventually assign ownership of the entire repo folder to apache user (www on my Mac server)
        - sudo -u www -s to test git in terminal as apache user… created a .ssh folder in /Library/Webserver and went through the process of setting up the ssh key pair for www and adding it to GitHub
        - also had to specify full path to git in php page ‘/opt/local/bin/git pull’ (or you can update the path for the www user)

  • http://www.szkolenia.eventis.pl Szkolenia

    A little bit complicated, can’t solve problem that Apache user cant exec a bash script too:(

  • http://wearemothership.com Stuart

    Great article. I am in the process of moving all our website and api source to GitHub.
    My question is about workflow with dynamic sites like blogs or web apps that also create content on the web server on the fly.

    Lets say I have a domain which has our website, blog and also some api’s which are used by our mobile app. Ideally i want that domain as one repo. (I had thought about one repo for the all domains/subdomains but probably not a good idea).

    So I start off with my local repo on my mac, I push that to GitHub and using a hook that gets pulled by the web server. So far this all makes sense. But what happens if content is created dynamically on the web server? (we have an api that creates some user files on the server). Is there a way to send those new files back to GitHub?

    What about a WP blog where a number of editors may be uploading images etc. Every time the server pulls from GitHub would i lose those new files?

    Sorry if these are silly questions. I am used to using SVN in Xcode for our app but so far all my web development work is done in Coda with a local copy on my Mac and I just sync to the server copy. You know, the usual way. So git is somewhat new to our workflow.

  • Mauro

    This tutorial needs a “sticky” to stay forever in frontpage.

  • Jackosn

    This looks perfect but I have been trying to get his to work for the past two days without any luck

    I can execute the file from command line (# php github.php) without it asking for a passkey otherwise nothing happens. I have the project in home/www/ Any ideas?

  • Chris

    Hey,

    Nice tutorial!

    When I try and run git pull from php, I get a ‘Permission denied’. Is this because the command is being run by the ‘apache’ user and not by ‘root’ who created the keys?

    If so, what is the solution?

    Thank you.

  • Gibbs

    Thanks for the great tut. This was exactly what I needed. Works great with Unfuddle btw.

    Things I learned along the way (Probably obvious but wan’t for me) :

    1. SSH user needed to be the primary user for my domain account on the server.
    2. This same user needed to set the SSH keys and run git init.
    3. Needed to turn off PHP Safe Mode so the script could run shell_exec.

    Simple I am sure, but completely forgot to look at the safe mode setting. It was a new sever setup and I failed to check php.ini.

    Thanks again.

  • http://waterstreetgm.org Terry Sutton

    This doesn’t (easily) work on shared hosting.

    I’ve got a Github account setup with USERNAME1, and my shared hosting account through Mac Highway was setup with USERNAME2. IE: All of my files are owned on the server by USERNAME2—the name assigned to me by my host.

    I really want this to work, but I don’t see a way around the two usernames.

    Any one solved this issue? I suspect that many of us here are on shared hosting.

    Thanks!

    Terry

  • Xander

    How to proceed when I do not have terminal?

  • http://menelikseth.com Menelik Seth

    Hi Jeffery!

    Thanks for breaking down github and SSH. The tutorial was very easy to read and follow, and it made for an excellent first step into the world of social coding.

    One puzzle:

    I got everything working, except for the last bit of Step 4. I can’t get github to trigger the pull.php file. When I click on it manually (by launching the URL) the pull request works, and my server is updated from the repo. But for some reason, github won’t trigger it automatically when I commit & sync.

    Any ideas?

    • http://www.facebook.com/njonas Nick Jonas

      I had the same issue, were you able to find a fix for this?

  • Will

    Thank you for writing this tutorial. Up to this point I was referencing:

    http://joemaller.com/990/a-web-focused-git-workflow/
    http://webxl.net/2011/03/10/managing-wordpress-with-git/

    This tutorial seems the most straightforward and allows integration with github which is fantastic.
    The website I was wanting to use this for is is a WordPress based implementation, so changes would be happening on server as well as on my own computer. To accomplish that I made the github.php script look like this:

    <?php
    `git pull`;
    `git add -A`;
    `git commit -m "Update: Website based changes"`;
    `git push`;

    Now everything is kept in sync. Coupling this with Cloud9 I can use any computer to edit my code anywhere. I welcome any thoughts on my addition. Thanks again Jeffrey.

  • http://www.ubezpieczenie-oc.pl Ubezpieczenie OC

    Great artice, it should be fasten on a main page;-)

  • http://citylig.ht Simon Robb

    Thanks Jeffrey, this is a brilliant tutorial.

  • Mat

    Hi!

    Nice article, I liked it a lot and put in use quite immediately.
    Anyhow I need to report that my hosting provider contacted e to strongly recommend not running shell scripts or commands from inside a PHP script.

    Second I would notice that on my hosting the `git pull` that pulls the git repo,
    reports progress and messages to stderr so them are showing up in the error log.

    Do you have any suggestion for avoiding this, without any third parties services and without double “push”?

  • Tzoreol

    Got this :

    “Host key verification failed.

    fatal: The remote end hung up unexpectedly”

    Somebody knows how to fix ?

    • http://twitter.com/heyallanmoreno Allan Moreno

      Same here :(

Newer Comments →