Getting Started with: Git, GitHub, and GitHub pages

Setting up git and GitHub for the first time

  1. Install git on your local computer by following the instructions here. Git is a code project management tool that is used primarily for version control and collaboration. The documentation is excellent.

  2. Make a GitHub account. You might consider applying to link your account to your school email via GitHub for Education. This comes with a lot of perks, like private repositories and free access to the GitHub Copilot plugin for VSCode and PyCharm.

  3. Create a local project folder on your computer containing code that you want to track. For example, you might want to create a folder for your course homeworks, or for labs, or for the final project. For now, let’s just make a test repository, to make sure that everything is working.

$ mkdir test_repo
$ cd test_repo
  1. You should now be inside your new repository. Add a README.md file to your local repository. You can use a text editor like VSCode, Jupyter Lab, or Sublime Text, or you can do this in the Terminal with your preferred editor. On macOS I normally use emacs (information here), but many may prefer nano or vim. If you want to use a different editor, replace emacs in all of the following steps with your preferred editor.
$ emacs README.md
  1. If this is your first time setting up GitHub and git, create a top-level file on your system called .gitconfig. Mine is located at the top level ~/.gitconfig and contains the following lines
[user]
        name = williamgilpin
        email = williamgilpin@gmail.com
[core]
        editor = emacs
[credential]
        helper = store

Of these lines, the [user] and [credential] fields are the most important fields.

  1. We now want to create the online repo, which is known as the remote repository. In your web browser, log into your GitHub account, and then make a repository with the exact same name as your local project folder. When prompted, do not initialize your online remote repo with a README or license. When the empty repository has been created, it should be located online.

    https://github.com/yourusername/test_repo

  2. If this is your first time using GitHub, you need to create Personal Access Token (PAT), which GitHub uses instead of passwords when you access a remote repo from th Terminal. Go to the GitHub website and make sure that you are logged in. As of writing, the web interface to create a PAT is as follows:

  • Go to github.com
  • Select your profile on the right side and go to: Settings > Developer Settings (bottom of left sidebar)

    Personal Access Tokens > Tokens (classic) > Generate New Token (classic)

  • You will encounter a screen where you name your token and then set granular permissions for the token. I chose to use a Classic token with no expiration, and I enabled full permissions for everything on the Token.
  • Hit “Generate Token” when you are ready.
  • You will be taken to a landing page with a list of your tokens. Copy the one you just created, which is likely a long string of letters and numbers. This will likely be the only token on this list.

If you run into difficulty, please follow the more detailed instructions provided by GitHub on created a PAT here. If setting up a PAT fails, then you might instead opt to authenticate with SSH. See the instructionshere

  1. Now return to your empty repo’s GitHub page, https://github.com/yourusername/test_repo. There will be instructions there listing what to do in order to get everything working on your local repo, but I’ve summarized them here. In your Terminal, navigate to your local repo. You will run the following commands in sequence:
$ git init
$ git add .
$ git commit -m "first commit"
$ git branch -M main
$ git remote add origin https://github.com/yourusername/test_repo.git
$ git push -u origin main

These commands first tell git to treat the directory as a git project with version tracking. The git add . command then adds all files or file changes. The branch command then confirms that you are on the primary project branch, and the git remote command actually initiates the connection to the remote repository you just made online with GitHub.

The final git push command sends your local changes to the remote repository. For your first push, GitHub will prompt you to provide a username, followed by a password. Instead of your GitHub.com password, please enter the PAT that you created in the last step.

If everything works, your Terminal will show a message like the following

(base) william@cns-f-pmaa59131 test_repo % git push
Enumerating objects: 7, done.
Counting objects: 100% (7/7), done.
Delta compression using up to 10 threads
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 60.62 KiB | 2.53 MiB/s, done.
Total 4 (delta 3), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (3/3), completed with 3 local objects.
To https://github.com/williamgilpin/test_repo
   e049541..ee2cb7d  main -> main
  1. Now that git is working, you can modify and update your local repository, and then manually push updates to your remote. To practice these steps, make some changes to your local repository; for example, by editing into your README.md file.
$ emacs README.md

Whenever you want to apply those changes to the remote (the GitHub version of your code), first add these files in the Terminal (make sure you are in your repository). Since we already made a README.md file, you will need to start by staging the changes in your local repository.

$ git add .
  1. Then commit the changes with a short but descriptive message
$ git commit -m "added example changes to the README file"
  1. Then finally send the update to GitHub
$ git push

Both your local and the GitHub versions of your repo will keep track of the sequence of commits you’ve applied, making it easier to roll back your changes at any time. You’ll notice that GitHub treats README.md as a special file—it renders it into a nice page, similar to a website’s index page, that represents the first thing a user sees when they look at your repo. Usually we want to put a description of the repo, dependencies, and a minimal working example into the README.md—although sometimes the README contains full documentation, graphics, etc. The Google Jax repository is a great example. Rather than HTML, the markup language used for README files is Markdown, which is like a blend of HTML and LaTeX with lighter syntax than either one. You can learn more from the Markdown guide or by looking at the unrendered version of this course’s own README file

You might have local files that you don’t want to appear in your public repo. For example, in the course repository I have solution files, as well as personal files like cached data, that I don’t want to appear on GitHub. For files of this nature, it’s usually a good idea to create a file called .gitignore in the top-level repository of your local repo. This specifies files or patterns that should be ignored. See the gitignore documentation as well as the one I’m using for the course repo

Create a website using GitHub pages

We are now going to a create a basic website to accompany our repository. This isn’t always necessary, but this feature is useful to quickly create a website—–for example, for the computational physics course webpage is written entirely in Markdown, and rendered into HTML automatically every time an update is pushed to GitHub (which also provides free hosting for small websites)

  1. The best approach is to follow the instructions from GitHub. Use the instructions for a Project Site.
  • Create a new git and GitHub repository, or go to an existing repository’s settings in the online GitHub GUI
  • In the Settings menu, go to “Pages” in the sidebar. Enable GitHub pages and, if prompted, pick a source as your main or master branch.

After some time passes, Github will automatically convert your README.md` file into an index.html file and then host it at:

https://yoursusername.github.io/my_repository

GitHub pages are structured as “actions” that run every time you push from your local to the remote on GitHub. This is similar to automated testing, which we will explore in a future lab. As a result, your website won’t automatically update as quickly as the repo itself, due to an extra “build” step. If the “build” fails, then on your repository you will see a red “X” on the commit. An orange dot indicates that the build has not occurred yet. If your website has still not updated, go to the URL and refresh the page with a clear cache; [CMD] + [Shift] + R

Collaboration

When used for single projects, GitHub acts sort of like a manual Dropbox folder, where you deliberately decide when to update the copy of your code that exists in the cloud. This might seem tedious, but as projects grow it becomes useful to keep track of versions, run automated tests before commits, check for conflicting commits, etc. One of the most important use cases is collaboration, where multiple people are working with the same remote repository.

  • Fork the repository on GitHub via these instructions
  • Make any changes that you want. It’s probably best to start with something small, like fixing small errors
  • Make sure that your local branch is up-to-date with the course using fetch (see instructions here). This is to account for the case where I’ve made changes to the course repo while you’ve been making your changes.
  • When everything looks good, submit a pull request using the instructions here

For smaller projects (like class projects), it isn’t necessary to use pull requests—instead, you can invite collaborators to a shared repository, and everyone will be able to make commits.

Important: if you have a fork that you are updating from a main—such as a fork of a class repository that you are using to complete homeworks–make sure that you don’t use your assignment fork to submit pull requests to my main repository. Since your versions of the assignments will override mine, it could lead to weird merge conflicts. If you fork a single version of an active repo (like a class repo), periodically git pull to get any changes. Just double check to make sure that to git stash any changes you’ve made locally (like assignments)

Step-by-step Guide to Submitting a Pull Request

  1. Fork the Repository. Visit the respository that you want to fork. For example, the cphy repository. Click on the “Fork” button in the top-right corner of the page and follow the on-screen instructions. This will create a copy of the repository in your own GitHub account.

  2. Clone Your Forked Repository

    • Open your terminal and run the following command to clone the forked repository to your local machine:
    git clone https://github.com/YOUR_GITHUB_USERNAME/cphy.git
  1. Create a New Branch. Navigate to the repository’s directory and create a new branch for your changes. A typical name for a branch might be development or bugfix
    cd cphy
    git checkout -b NAME_OF_YOUR_BRANCH

  1. Make Your Changes. Make and save your changes to the necessary files within the repository.

  2. Commit Your Changes. Add your changes to the staging area, then commit them with a descriptive message:

    git add .
    git commit -m "Your descriptive message here"
  1. Push your changes to your forked repository on GitHub:
    git push origin NAME_OF_YOUR_BRANCH
  1. Create the Pull Request on GitHub. Navigate to your forked repository, and click on the “Compare & pull request” button. This will take you to the page where you create the pull request. After you have filled out the necessary information, click on the “Create pull request” button.
    • Go to the “Pull requests” tab on the original repository to which you are submitting a PR
    • Click the “New pull request” button.
    • Look for the link that says “compare across forks,” and select your fork and the branch you created. The other drop-down menu should be the main branch of the original repository.
    • Click “Create pull request”.
    • Fill out the pull request template with all the necessary details.
    • Click “Create pull request” to submit your pull request for review.

Some repositories will include a CONTRIBUTING.md file that gives guidance for contributors submitting pull requests.

Link your local machine to GitHub account via SSH

GitHub no longer supports password authentication. Instead, you can use SSH keys or a personal access token (PAT). I prefer SSH keys, because they are a bit more general and can be used for other things.

In terminal,

    ssh-keygen -o -t rsa -C "username@email.com"

Press enter twice to assign to default location and have no password. Now view the SSH key by going to

    cat ~/.ssh/id_rsa.pub

Copy and paste this into your SSH key collection on the GitHub website. To test that this is set up correction, on your local machine run

    ssh -T git@github.com

and receive response

    Hi username! You've successfully authenticated, but Github does
    not provide shell access.

A nice guide is here

Fork a repo online and then submit a pull request {pull-request}

  1. Create a local clone of the target repository

    git clone http://github.com/williamgilpin/repo_name

If you are repeatedly prompted to authenticate, but run into issues because GitHub no longer accepts passwords, you can use the git protocol instead of https. This requires that you have an SSH key set up (see above). To do this, run

git clone git@github.com:williamgilpin/repo_name.git

If this still fails, then check that you have SSH set up correctly. See the instructionshere

  1. Now fork the repository to your own GitHub account. Go to the original repository on the GitHub web interface and click the “Fork” button.

  2. Add your fork as a remote:

    git remote add myfork https://github.com/YOUR_USERNAME/repo_name.git
  1. Create a new branch to work on:
    git checkout -b newfeature
  1. Make your changes, and commit them:
    git add .
    git commit -m "Added newfeature"
  1. Push your changes to your fork:
    git push myfork newfeature

  1. On the GitHub web interact, create a pull request from your newfeature` branch to the original repo’s main branch.

  2. If the original maintainer requests changes, or you make further changes, commit them to your newfeature` branch. Then push to your fork again. The PR will automatically update.

  3. Once approved and merged, you can delete your local branch and pull the updated main branch:

    git branch -d newfeature
    git checkout main
    git pull origin main

Create a new feature

On your local copy of the repo, make sure that you are updated to the latest version of the remote

    git pull

Now create a new branch locally

    git checkout -b [name_of_your_new_branch]

Now push to the remote

    git push origin [name_of_your_new_branch]

Now make edits to your local version, pushing to the remote as needed. Occasionally, you may also need to pull the latest changes from the remote main.

Editing commit history

To alter or combine the last four commits, run

   $ git rebase -i HEAD~4

A text editor will pop up. Replace “pick” with “squash” for the commits that you want to merge together. It will then prompt you to come up with a new commit message for all of the commits that you just squashed.

Save and close using standard emacs commands

If you’ve already commited, you have to force the update:

  $  git push origin master --force

Remove a “dirty” commit containg a large or private file

If you commit a large file that is rejected on push (for example, GitHub will reject files >100MB), you need to completely remove it from the commit history before you will be able to push again. This is a also good idea if you have committed private file that you don’t want to be public.

Install bfg

    brew install bfg

Remove the file from the history. Don’t specify the path, just the filename (e.g. my_bad_file.zip)

    bfg --delete-files my_bad_file.zip

Fix the commit history to remove this bad file

    git reflog expire --expire=now --all && git gc --prune=now --aggressive

If using bfg fails, try overwriting the hashes. This can cause issues on shared repositories, and should be used as a last resort.

    git filter-branch --force --index-filter "git rm --cached --ignore-unmatch <path/filename>" --prune-empty --tag-name-filter cat -- --all
    git push origin master --force

Examine and merge a pull request

After recieving a pull request, make sure your local copy of the repository is up to date with master and that you’ve committed all changes. Now,

	git checkout -b otherusersname-master master
	git pull https://github.com/otherusersname/pypdb.git master

Now run tests, make sure everything appears to be working. You can also make any edits to the documentation, etc on this branch. Once you are satisfied (and if there are no conflicts), merge this branch:

	git checkout master
	git merge --no-ff otherusersname-master
	git push origin master

If you get an error when switching branches, you might need to discard some local changes to master (do this carefully). In this case, use the force flag

    git checkout -f master

Fork a repository online

You can fork a repository and make basic changes from the online GitHub GUI. After forking and making any changes online, you can get a local copy by running

git clone https://github.com/username/repo_name

Detailed instructions here

Update local with changes to remote

    git pull origin

This combines a fetch with a merge. If you want to do these separately, you can run

    git fetch origin
    git merge origin/master

If you have uncommitted changes, you may need to stash them before pulling

    git stash
    git pull origin

You can then add your changes back

    git stash pop

Force overwrite local with remote

	git fetch origin
	git reset --hard origin/master

Downloading a remote repository without forking it

Sometimes you just want to download a copy of someone’s code without collaborating, forking, etc. For situations like these, you can use git clone. I recommend cloning from the HTTP address rather than the git// address of the repo

Installing a Python Package from GitHub

If there’s a setup.py file in the repo, you can install using pip

    pip install git+git://github.com/someusername/somerepo

Modifying commit history

To alter or combine the last four commits, run

    $ git rebase -i HEAD~4

A text editor will pop up. Replace “pick” with “squash” for the commits that you want to merge together. It will then prompt you to come up with a new commit message for all of the commits that you just squashed.

If you’ve already commited, you have to force the update:

    $ git push origin main --force

Forking a repository summarized

clone forked repo locally

    $ git clone "https://...MY_USERNAME...

add upstream branch

    $ git remote add upstream "https://...THEIR_USERNAME...git

make a new branch

    $ git add branch BRANCH_NAME

switch to new branch and make edits

    $ git checkout BRANCH_NAME

push new commits

    $ git add .
    $ git commit -m "test commit plz ignore"
    $ git push

go to github and make a pull request

Errors

Cannot stage changes

Sometimes instead of git add . you need to use git add --all This can be fixed by stashing and then immediately un-stashing:

    git stash
    git stash apply

Permission issues

error: insufficient permission for adding an object to repository database .git/objects

Somehow the ownership got messed up for some files. From project base directory, try running

    cd .git/objects
    ls -al
    sudo chown -R yourname:yourgroup *

yourname and yourgroup can be figured out by seeing what the majority of of the ls -al usernames and groups are. My “group” appeared to be staff for some reason. This answer is taken from StackExchange

Recieve warnings about passwords being deprecated

After using GitHub from the command line, I recieved the following email

    Hi @williamgilpin,

    You recently used a password to access the repository at williamgilpin/dysts with git using git/2.24.3 (Apple Git-128).

    Basic authentication using a password to Git is deprecated and will soon no longer work. Visit https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information around suggested workarounds and removal dates.

    Thanks,
    The GitHub Team

I followed GitHub’s instructions here to create a PAT.

To store the PAT after creating it, I follwed the instructions

  • here
  • and then here
  • I also removed the file ~/..git-credentials
  • Other useful information about storing PAT here and here

Transfer repo to an organization

Transfer as normal using the “Settings tab”

Global Configurations

My ~/.gitconfig file is as follows

    [user]
            name = myname
            email = myemail@email.com
    [core]
            editor = emacs
    [credential]
            helper = store
    [alias]
            acp = !git add . && git commit -m "latest" && git push

Removing git-lfs

Oftentimes, you might find yourself needing to completely remove git-lfs from a respository, and then re-add the files it tracks to git. Remove all traces of git-lfs from .gitattributes. Then, from the command line, run

    git lfs uninstall

You will likely still need to manually go through and untrack every file that was previously tracked by git-lfs. This can be done by running

    git lfs untrack filename

You can list out all tracked files by running

    git lfs ls-files

Deprecated: connecting a new computer to GitHub using password authentication

These instructions no longer work, now that GitHub dropped support for password authentication.

To use traditional authentication, in Terminal,

    git config --global user.name github_username
    git config --global user.email my_email@email.com
    git config --global core.editor emacs

The last line sets the default editor to emacs. The first time you push changes to remote, you will be prompted for your password. This will be saved for future use

If you are still repeatedly prompted for your account credentials, use

    git config credential.helper store

Note that running the above will cause an unhashed copy of your GitHub password to be stored locally.

I receive the following “error: unable to read askpass response from ‘/usr/libexec/openssh/gnome-ssh-askpass’”

Run the following command to fix this issue

    unset SSH_ASKPASS