How To Add Content

This page provides specific practical detail on how members can add content to the club web site

Introduction

It is in the member directory of the web framework files that you can access and easily update.

Don’t worry if you have never worked on a website before. We use a very popular and easy to use template system with markdown that you can gradually get familiar with and use a collaborative system that automatically builds and deploys when you signal you are ready.

Markdown, which is like a web page shortcut language, is used to write file contents. The basics can be mastered very quickly and it is very good at keeping writing organised with links, pictures and tables (if you want).

We have added directories to member, /member/. You can use these directories and files as a starting point but please do not use a directory or files someone may have started working on.

What follows is a detailed point by point description of how to add a new directory to /member/photosmember/ that you can claim as your own and that will be indexed by /member/photosmember/

The workflow below appears confusing at first. However it is a well tried and trusted system that is very practical and after getting used to it you will fight like crazy against using any other way!

Getting Access

  1. Come into the club and talk to the president, Michael. Tell him what you are interested in and he will introduce you to the goals of the website and its guidelines. Don’t worry! The goals of the website are closely aligned with the goals of the club.
  2. If you still want to go ahead and the president agrees then obtain a free gitlab account from gitlab.com. All you need to provide is an email contact address and choose a username. DO NOT ENTER CREDIT CARD DETAILS AND DO NOT CLICK ON ‘START TRIAL’.
  3. Email the club with your gitlab username, stating the club has approved you as a club website developer. Even if you have never developed a website before we will still consider you a website developer. It is the intention that counts
  4. The club will add add your gitlab username to its list of website developers on gitlab and inform you.

Getting Help

Once you have access you will be able to access the web developer wiki. You need to enter in your gitlab login credentials. The wiki contains the most up to date version of the CMLC Web Developer Guidelines.

You can get help, raise issues, make suggestions, participate in discussions and communicate with other web developers as a group at the Issues Board for club website developers.

We want you to succeed. If you need help and you say nothing then we cannot help you. The issues board is where you will get help if you raise a new issue asking for help.

Software Required

If the following instructions are too confusing then we can help show you at the club or you can ask to join a workshop. If you bring a laptop to a workshop then we can help you set up your laptop at the workshop.

  1. To view edits on a local webserver and optionally create blank template pages install the extended version of hugo. As mentioned, make sure hugo is added into the PATH environment variable.
  2. To fetch files and push changes with Windows install git
  3. Use any editor.
    • An editor with convenient colour coding of markdown in Windows is Notepad++.
    • A more complicated but popular editor which will only provide a partial preview of markdown code as it will appear on website is Visual Studio Code. There is a way to get a better website preview using command ‘hugo serve’ (see below). In Windows a right click will show a menu to open a directory in ‘Visual Code’.

Optional Software

Imageresizer and mogrify resize jpeg images. Jhead fixes rotation and strips metadata from images. Handbrake transcodes resizes (or transcodes) videos.

Wsl Debian is a UNIX type environment in Windows most suitable for running UNIX type cli tools. More information following.

GUI means Graphical user Interface. CLI means Command Line Interface. For wsl Debian a mix of Windows and wsl Debian software can be used

  • For Windows: ImageResizer (gui), jhead (cli), handbrake (gui or cli),
  • For Windows wsl Debian: ImageMagick for mogrify (cli), jhead (cli)
  • For non Windows Debian with GUI: as above with handbrake (gui or cli)
  • For macOS (untested): as above.

The large files section has practical information on keeping video sizes down.

Wsl: a different approach for Windows users with plenty of support online for beginners

There is another approach for Windows users, wsl, that is more complicated at first but has plenty of support for beginners, makes it easy to install software, use software as commands, can be easily scripted, is easy for others to help you with and so has enormous productivity benefits.

Wsl is now easy to install with up to date Windows 10.

After enabling wsl in windows and installing Ubuntu or Debian from Windows store:

To open a console in any directory from Windows Explorer: hold shift key down, right click in directory, click on ‘Open Linux shell here’.

A quick way to get going, tested with Debian from Windows Store (can be easily scripted):

sudo -s
apt update
apt upgrade
apt install git curl imagemagick jhead
# Download hugo separately to get necessary latest updates.
# An alternative is to exit now, extract/move hugo to ~/bin instead 
# and place the line 'export PATH=~/bin:$PATH' in file ~/.bashrc.
curl -L -o hugo.tgz https://github.com/gohugoio/hugo/releases/download/v0.61.0/hugo_extended_0.61.0_Linux-64bit.tar.gz
tar -zxvf hugo.tgz
mv hugo /usr/local/bin 
rm LICENSE  README.md hugo.tgz
exit

#choose directory
git clone https://username:password@gitlab.com/cairnsmlc/website.git
cd website
hugo serve  # have look around on http://localhost:1313 

#open another console in website/content/member/photosmember/somedirectory
mkdir gallery
cd gallery
# more information below on commands following
# place jpg images in directory
# Do a Ctrl + C on console with 'hugo serve', as the following will result in a lot of unnecessary activity otherwise. Start up 'hugo serve' again when finished.
mogrify -resize 320x569 * # use instead of ImageResizer for windows
jhead -autorot *
jhead -purejpg *

#usage note for Visual Code editor in non wsl Debian. To start Visual Code within directory, use command 'code .' (full stop following) instead of right click.

Using software to create, edit and view your page

This is specific to Windows

  • With Windows Explorer, choose any directory with right click, choose ‘Git Bash Here’ to open a console
  • Suppose username is your gitlab username and password is your gitlab password then in console enter

    git clone https://username:password@gitlab.com/cairnsmlc/website.git
    cd website
    hugo server
  • You only need to use the git clone command once. You can update the clone with command following in the website directory, as shown later with git pull --rebase.

  • You now have access the club website served on your PC or laptop at address localhost:1313. You can stop the server wth Ctrl+C. Sometimes it is a good idea to stop the server and restart it again with ‘hugo server’ command

  • With Windows explorer and in the website directory right click again and choose ‘Git Bash Here’

  • Check you are in website directory with

    pwd
  • Create a new username directory and index.md file in content/member/photosmember directory. In the example below note the content directory is left out of the optional hugo command

    hugo new member/photosmember/username/index.md 
  • Now edit the index.md file, including title, with your editor. Look at other files in website for examples or look at examples on the web using markdown. You can drop image files into your directory but please make sure they are no larger than 100Kb each and there are not too many. If you do need to upload very large files or a lot of files then please contact web admin who will provide a solution for you. More information below.

  • View how the page will look at localhost:1313/member/photosmember/username/

Writing content with markdown and shortcodes

The most complete reference relevant to this site is at writing content

Specific examples of shortcodes for youtube and images

This section deals with shortcodes (youtube, figure and gallery) for showing youtube videos, constrained single images and images presented as a gallery from a directory. The figure shortcode for images is safe to use with landscape images. The gallery shortcode can cause issues with landscape images on mobile phones but this can be fixed by passing a parameter.

Example Linking to a youtube video

If you view Trevor Hannam’s faceting video on youtube you can see it has an url at the top of webpage of youtube.com/watch?v=rdCVqqt6Jqc

If you take the code after the = sign, rdCVqqt6Jqc, you can embed the youtube video with the following code

{{< youtube rdCVqqt6Jqc >}}
and get the following result:

Example showing all images in your directory with resizing, rotation fixing and metadata removal

If you place jpg images in a gallery subdirectory of your directory then the following code will show all of them at once:
{{< gallery >}}
Clicking on an image will bring up the full image

For example see /member/photosgroup/electroforming/ that is linked from /member/photosgroup/.

The last line of the
/member/photosgroup/electroforming/index.md
file is

{{< gallery >}}

You are not limited to using gallery as the directory name and a page can show multiple galleries. There is an example in /member/photosgroup/images/index.md (as shown below).

Also you can ensure captions are shown when images are shown from a gallery. There is an example in /member/photosgroup/chudleigh park 2019/index.md (also shown below).

Please only use jpg files for images. They are suitable for resizing and display. The following applies to jpg files.

All jpg images in a gallery directory can be resized easily resized easily with the download from https://www.bricelam.net/ImageResizer/

jpg

In windows, enter the gallery subdirectory (by clicking on any file within directory), use keyboard combination Ctrl with A to select all photos, right click on selection and choose ‘Resize Pictures’ to bring the dialog box above up.

This software will help total file size in your directory under 10MB. If each image file was on average under 100KB then 100 images would be under 10MB.

Unfortunately the checkbox ‘Ignore the orientation of pictures’ has no effect on the issue following.

To ensure images are shown with the correct rotation it is necessary to convert images with non zero exif rotation metadata and re-save them. An easy way to do this is with command line tool jhead from http://www.sentex.net/~mwandel/jhead/

In the gallery directory:

jhead -autorot *.jpg

After fixing rotation, all exif metadata should be removed:

jhead -purejpg *.jpg 

If you have landscape images then you can avoid an irritating problem on mobile phones with the gallery shortcode by including an automax parameter set to true

{{< gallery automax=true >}}

instead of

{{< gallery >}}

With landscape gallery images on mobile phones a page can load as zoomed in without the automax parameter. When zoomed in the menu at the top will scroll in and out instead of remaining fixed.

This /member/photosmember/hardcastle_michael/ page includes the following code to show a landscape image, seven gallery images and another landscape image. All the seven galley images are in a directory named gallery. This stripped code works well on a mobile phone and uses the gallery shortcode with no parameters.

{{< figure src="cabochons_silverbacked.jpg" title="Seven images of silver backed cabocons, including prizewinner, top left on sash, Cairns Show 2019" lightbox="true" >}}

{{< gallery >}}

{{< figure src="cabochons.jpg" title="Image of another eight cabochons" lightbox="true" >}}

Example of multiple galleries on one page

This /member/photosgroup/images/ page has more than 200 images broken into multiple galleries using the following code. The page code looks complex however it is very simple. All the images are in four different directories: club, works, samples and fossicking.

Views of finished work, work in progress, mineral samples and field trips.

Presented so you can view and scroll though quickly.

## Club Activity Photos
{{< gallery album="club" >}} 

## Gems, Polished Stones, Silverwork Photos
{{< gallery album="works" >}} 

## Mineral Samples Photos
{{< gallery album="samples" >}} 

## Fossicking Photos
{{< gallery album="fossicking" >}} 

The /member/photosgroup/chudleigh-park-2019/ has a single galley directory but clicking any image will show the image with a caption. The nine images in the directory are named 01.jpg to 09.jpg to control the order they will appear in the gallery (alphabetical order). The shortcode is the same:

{{< gallery >}}

However the ‘front matter’ of the page, that appears between the following marks at the top of the page

---

has the following code:

gallery_item:
- album: gallery
  image: 01.jpg
  caption: Have chair, will fossick
- album: gallery
  image: 02.jpg
  caption: Is this one?
- album: gallery
  image: 03.jpg
  caption: This is what you are looking for! 
- album: gallery
  image: 04.jpg
  caption: Creating a dust storm
- album: gallery
  image: 05.jpg
  caption: The shaking bottle
- album: gallery
  image: 06.jpg
  caption: Found a likely specimen
- album: gallery
  image: 07.jpg
  caption: Air conditioning
- album: gallery
  image: 08.jpg
  caption: Teaching them young
- album: gallery
  image: 09.jpg
  caption: Here is a beauty

Importance of adding in a trailing slash for defaults. Requesting shorter URLs

To keeps costs down the club is using static file storage, S3, not a regular web server. S3, on its own, does not recognise the directory part of a name as a directory. For S3 the directory part of a name is part of the file name.

Steps have already been put in place to make sure any URL ending in / is replaced by /index.html. Without a trailing slash a directory name is not recognised. Until we implement and thoroughly test a solution you should assume a slash is required at the end of each directory name. For example notice the trailing slash at end of https://cairnsmineral.club/member/addingcontent/

You can request we add in directory aliases to shorten URLs.

Updating the website with your changes

This is the simplest part

  • With Git console above make sure you are still in the website directory
  • Make sure you do not have files, such as a large image file or any notes that you do not want uploaded in the webserver directory or subdirectoriesbe asked
  • If you have not run these command then you will be asked by git to fill in your email and name and told how to. You will not be asked again
  • Before starting editing it is a good idea to run

    git pull

    in case files have been altered by some one else

  • Run the following commands in one go after you have finished your edits from the website directory. Remember this is a collaborative system so their are potential conflicts. The system is designed to avoid conflicts and to help you resolve conflicts when they do occur. Most of the time you will not have conflicts to resolve. The first command tells git you want to add in changes to your local working copy of the file repository (repo). Remember the full stop at the end of the first command. The second command copies across your working copy changes to the local clone. The third command updates the local clone with any changes that have occurred and rewrites your commit history in such a way to make them easier to follow on top of the clone. The fourth command uses your local clone to write your changes to the repo on gitlab and triggers a rebuild with deploy

    git add .
    git commit -m 'short relevant message about purpose of your changes'
    git pull --rebase
    git push
  • That is it. After a minute or so while the club website files are rebuilt on gitlab and deployed from gitlab as far as the club’s S3 bucket.

  • Since there can be up to a 24 hour delay before your changes will be visible on the normal club URL https://cairnsmineralclub.rocks, you can instead view your changes on an URL that bypasses the CDN (cloud distribution network) to go direct to the S3 bucket. This URL is http://static.cairnsmineralclub.rocks. Note the use of http instead of https. You will also need to remember that this URL does not directly access the S3 bucket. Instead it goes through an endpoint that does interprets S3 file names as having a directory structure, so for example http://static.cairnsmineral.club/member/addingcontent (no trailing slash) will work but https://cairnsmineral.club/member/addingcontent (no trailing slash) will not work, requiring https://cairnsmineral.club/member/addingcontent/ (trailing slash) instead. A fix will be put in place for this.

Dealing with a rare merge conflict

On the rare occasions you get told there is a merge conflict the simplest solution for beginners is the following.

  • Store a copy of the file or files causing the conflict outside the website directory. You will be told what these files are.

  • Run the following commands

    git fetch --all  
    git reset --hard origin/master
  • Since this will remove your changes causing the conflict, put back in what you want using the files you stored and continue on as before but do not overwrite the files. Instead edit in the changes you wanted to make to fit in with a collaborative style.

Working with a lot of files or very large files

  • The club has a policy of keeping large files off the git repo. One of the reasons for this is help keep git based offline editing working smoothly. So you may find club admin has reduced the size of your image files or has moved files in your directory elsewhere and altered links. Generally if your directory file contents are under 10MB in total then the directory file contents will be unaltered and/or left where they are. For this reason it is a good idea to use git pull before new edits in case your files have been altered. You probably will not notice if your files have been altered if just image sizes are reduced. Use git pull --rebase after you have made edits (as indicated above)
  • The club will allow very large files or groups of files to stored for access from the website, such as files over 100MB. For example if you have video files you do not want to go on youtube and link to. It is best if this is discussed with club admin beforehand so they can upload them separately and provide you with links to be used in your web pages.
  • A convenient way to make a large file (such as a video file) available to someone to add into the website is to add the file to your own Google Drive and then from Google Drive share it with that other person. Google Drive is available form the App Store for iPhone and makes it easy to share large files with Android users and with PCs.
  • A convenient way to make a large number of files available is to send them all to a single directory of Google Drive and share the directory with someone.

Keeping video files under 1MB

If you can keep video files under 1MB then keep them in your directory. If you keep to an advised policy of not allowing any video to be longer than 30 seconds and you use handbrake software for conversion to web optimised MP4 with a 480p30 setting and quality set to 30 RF then you can expect a converted .MOV file to reduce in size by typically well over over 95% to under 1MB.