Keep My GitHub Fork Up-to-date

These only show you how to update the local copy. And, it’s assumed you’ll ONLY be merging the branch you are working on.

Christina’s Way:

She forgot to tell you that master has to be the locally checked out branch.

Also, the only two branches which will be merged are the origin/master and the upstream/master. The rest of the branches will hang around looking like duplicates in your GitHub.app GUI.

GitHub’s Way

Narf’s Cleaner Way

The part about cloning and changing the current directory, etc. are the same. But, instead of adding and fetching you’ll do:

This way you don’t get the duplicate branches (which are empty anyways.)

Narf’s advice about (other way) & (contribution)

Advertisements
Image | Posted on by | 1 Comment

Using the Bash CLI


Basic Bash


Command line editing is covered elsewhere.


$ exit

Exit the shell. If the shell you are in has a parent shell then you will end up in it. If the shell you are in is the login shell then you can’t use the exit command. You’ll have to use logout.

Ctrl+D (which is the key sequence for EOF) is another way to exit the shell.


$ export EDITOR=vi VISUAL=vi

Sets the variables EDITOR and VISUAL.

It may be better to have these values set to the complete path for vi on your system.

export — specifies that those two variables are to propagate to subprocesses — thus becoming environment variables. In other words they are valid for this shell, a sub-shell, or a script you may run from the them. Child processes cannot export variables back to the parent processes which spawned them.

Although the built-in environment variables propagate to subprocesses, the shell must be explicitly told to do so with other variables, options, aliases, etc. The export command takes care of this — a more permanent way is to place their exported assignments in a bash configuration file (see below.)


source

$ source commandlist

executes comandlist/(a Bash script)


alias

$ alias search=grep
$ alias cd_www='cd ~/sites/testing/www'
$ alias la='ls -al'
$ alias printdir='pr * | lpr'

When using an alias in a command line bash makes a textual substitution of the alias for that which it is aliasing.

Aliases can be used only for the beginning of a command string. You can’t simply alias a directory path then cd to it. However, bash has a way around this problem. If the value of an alias ends in a blank, then bash tries to do alias substitution on the next word on the command line.

Aliases are recursive — it is possible to alias an alias. However, bash has protection so that something like alias ls='ls -al' will not create an infinite loop. If the word to be replaced is the same as the alias name (of the current alias) then no text substitution takes place.

If you type alias name , the shell will print the alias’s value. If you type alias without any arguments, you get a list of all the aliases you have defined. The command unalias name removes any alias definition for its argument.

Aliases are good; but shell scripts and functions are better. You’ll need to know the order of precedence when an alias and a function have the same name.


Setting Bash Options

Options let you change the shell’s behavior. The basic commands are set -o optionname and set +o optionname. The - turns the named option on, while the + turns it off.

Option Description
emacs Enter emacs editing mode (on by default.)
ignoreeof Don’t allow use of a single CTRL-D to log off; use the exit command to log off immediately. This has the same effect as setting the shell variable IGNOREEOF=10.
noclobber  Don’t allow output redirection (>) to overwrite an existing file.
noglob Don’t expand filename wildcards like * and ?
nounset Indicate an error when trying to use a variable that is undefined.
vi Enter vi editing mode.

There are more options; And some options can be set just using a dash and letter following the set keyword.

set -o — List all options with their settings.

shopt is a newer built-in for configuring the shell. It is meant as a replacement for option configuration originally done through environment variables and the set command.

shopt options option-names is the format for this command. You can find tables which list both the options and the option-names.

cdable_vars — is a notable option-name. If set, an argument to the cd built-in command that is not a directory is assumed to be the name of a variable whose value is the directory to change to.


Variables

Bash has some already built-in variables. Others you will define on your own.

varname=value — is how you declare/assign a variable.

If the value is more than one word, it must be surrounded by quotes. To use the value of a variable in a command, precede its name by a dollar sign ($).

$ echo "The value of \$varname  is \"$varname\"."

This works pretty much like it does in PHP. Single quotes (instead of double quotes) result in less interpretation. Also, you can omit the quotes entirely — although this is not advisable — because white space will be consolidated as everything is seen as a word separated by white space.

You should put double quotes around what you want to echo out — even if what you’re outputting is a single variable — the reason for this is to preserve the white space in the value of the variable.

$ LsOutput=`ls -al`
# Store command output in a variable using backquotes.
# For more on this see topic: Command Substitution

OneVar=$AnoterVar
# Assign the value of one variable to another.

You can do this:

$ AB=/usr/dog/contagious/ringbearer/grind ; export AB
$ cd $AB

Two reasons why braces are used when dereferencing variables:

Reason 1

If you want to have other text right up against the output from an environment variable, surround the variable in braces. This protects the variable name from being misunderstood. For example:

$ echo ${AB}/adventure
/usr/dog/contagious/ringbearer/grind/adventure

Reason 2

Because that’s the normal way to dereference a variable. $varname is the exception — and $varname doesn’t work in many situations. Here’s a list of those situations:

  • if your code refers to more than nine positional parameters: you must use ${10} for the tenth instead of $10.
  • the variable name is followed by a character that is a letter
  • the variable name is followed by a character that ist a digit
  • the variable name is followed by a character that is an underscore

unset — If you decide that you no longer want a variable to be set, you can use the unset command to erase its value. For example, you could type unset XYZ, which would cause XYZ to have no value set.


Prompt

PS1 determines your main prompt. PS2 determines your secondary prompt — and so on… If the shell needs additional input, it uses the value of PS2.

PS2 is called the secondary prompt string; its default value is >. It is used when you type an incomplete line and hit RETURN, as an indication that you must finish your command.

On my Mac this is what I have:

PS1 '\h:\W \u\$ '
PS2 '> '

Some prompt component escape sequences:

Command Meaning
\d The date in "Weekday Month Day" format
\e The ASCII escape character (033)
\H The hostname
\h The hostname up to the first "."
\n A carriage return and line feed
\s The name of the shell
\T The current time in 12-hour HH:MM:SS format
\t The current time in HH:MM:SS format
\@ The current time in 12-hour am/pm format
\u The username of the current user
\w The current working directory
\W The basename of the current working directory
\# The command number of the current command
\! The history number of the current command
\$ If the effective UID is 0 print a #, otherwise print a $
\nnn Character code in octal

PATH

BTW: I use the word “enter” to mean submit something to the shell.

This section is about:

Where does Bash look when you enter a command?

A command is a word you enter which correlates to the name of either (an executable file) or (a shell built-in construct.)

sameh-labibs-imac:~ samehlabib$ echo $PATH
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin:/usr/local/git/bin:/opt/local/bin

This show the default path for a user (since it has not been customized for samehlabib.) Directories in the path list are separated by colons.

If you put an (executable file)/command in one of these directories, you can use the command just by entering it.

The order of path directories matters. Directories are checked going left to right.

type is a useful command in this regard.

sameh-labibs-imac:~ samehlabib$ type vi
vi is /usr/bin/vi

If a command lives in multiple PATH directories, you can use type with the -a option to list all the directories where the command lives.

Also, the type command shows just the command’s name and its type if it’s a built-in command (like echo), an alias, or a function.

If you often use directories of commands that are not in your PATH, you can permanently add them. To do this, add a new PATH variable to your .bashrc file. For example, to add a new directory called /dirname/bin to your path, add the following line:

$ PATH=$PATH:/dirname/bin ; export PATH

This first reads the current path directories into the new PATH then adds the /dirname/bin directory.

CDPATH

CDPATH does for directories what PATH does for commands. It makes using the cd command easier.

$ CDPATH=:~/fr/lang/clause/long

The colon in the beginning allows a directory to be found in the current directory.

Bash provides something else to help cd to a directy; if you set the shell option cdable_vars , any argument you supply to the cd command that is not a directory will be looked at as a potential variable. Hence you can store long directory paths in variables to avoid re-typing them.

This allows us to omit the dollar sign in front of the variable when applying cd to it.


(other) Environment Variables

Variable Its Value
HOME Path to your home directory
SECONDS Number of seconds since the shell was invoked
BASH Path of currently running shell command
BASH_VERSION The version number of the shell
BASH_VERSINFO An array of version information for the shell
PWD Current directory
OLDPWD Previous directory

Configuration Files

~/.bash_profile Here you will put lines which specify the values for intrinsic environment variables you want to change and commands which run or produce output when you log in. Also, here you’ll source your .bashrc if you want it’s content to apply to the login shell.
~/.bashrc Is referred to as the environment file. Use it to specify definitions for things in your environment (Not for intrinsic environment variables though but for variables you want to become environment variables, option and aliases.) This file is automatically read when a subshell is started. However, to have this file apply its content to the login shell you’ll need the line source .bashrc at the end of .bash_profile
/etc/profile .. is the global .bash_profile
/etc/bashrc .. is the global .bashrc
~/.bash_logout .. is read and executed every time a login shell exits.

.bash_profile, is read by bash every time you log in.

.bash_profile is read only by the login shell. If you start up a subshell, it will attempt to read commands from the file .bashrc. You can use the source command from within .bash_profile to execute .bashrc.

Some variables you can assign in .bash_profile:

PATH Make sure to include the existing path in your assignment to PATH.
SHELL SHELL=/bin/bash
EDITOR EDITOR=/usr/bin/vi
PS1 PS1='\h:\W \u\$ '
PS2 PS2='> '

History


fc for history

You don’t need fc if you use the history features of the vi command line editing.

fc is a built-in shell command.

The -l option to fc lists previous commands. Arguments:

  • two arguments, they serve as the first and last commands to be shown.
  • one number argument, only the command with that number is shown.
  • a single string argument, it searches for the most recent command starting with that string; then it shows everything from that command to the most recent command.
  • no arguments, you’ll see the last 16 commands entered. bash also has a built-in command called history ; which does the same thing.

-n suppresses the line numbers.

-s allows you to rerun a command. See man page to understand the arguments you can use with the -s option. Among other things this allows for string replacements.

When used without options fc loads the default editor. fc runs the command(s) after you edit them.

  • With no arguments, fc loads the editor with the most recent command.
  • With a numeric argument, fc loads the editor with the command with that number.
  • With a string argument, fc loads the most recent command starting with that string.
  • With two arguments to fc, the arguments specify the beginning and end of a range of commands, as above.

Although this may look like a good way to generate a shell script, a better way would be to direct the output of fc -ln to a file — then edit that file and execute the commands when you’re happy with them.


I skipped History Expansion. This Bash history stuff is too involved!


History Environment Variables

Variable Meaning
HISTCMD The history number of the current command
HISTCONTROL If set to the value of ignorespace, lines beginning with a space are not entered into the history list. If set to ignoredups, lines matching the last history line are not entered. Setting it to ignoreboth enables both options.
HISTIGNORE A list of patterns, separated by colons (:), used to decide which command lines to save in the history list. Patterns are considered to start at the beginning of the command line and must fully specify the line, i.e., no wildcard (*) is implicitly appended. The patterns are checked against the line after HISTCONTROL is applied. An ampersand (&) matches the previous line. An explicit & may be generated by escaping it with a backslash.
HISTFILE Name of history file in which the command history is saved. The default is ~/.bash_history.
HISTFILESIZE  The maximum number of lines to store in the history file. The default is 500. When this variable is assigned a value, the history file is truncated, if necessary, to the given number of lines.
HISTSIZE The maximum number of commands to remember in the command history. The default is 500.
FCEDIT Pathname of the editor to use with the fc command.

Snippets


$ touch apple banana grape grapefruit watermelon

Is an example of how to create some empty files.


$ find /usr -print > /tmp/allusrfiles &

This command finds all files on your system (starting from the /usr directory), prints those file names, and puts those names in the file /tmp/allusrfiles. The ampersand (&) runs that command line in the background.


$ vi $(find / -print | grep xyzzy)

This vi command opens all filenames for editing (one at a time) that include xyzzy. This particular example may be useful if you knew that you wanted to edit a file for which you knew the name but not the location. As long as the string was fairly uncommon, you could find and open every instance of a particular filename existing in the file system.


$ echo "I am $[2002 - 1957] years old."
I am 45 years old.

In this example, the shell interprets the arithmetic expression first (2002 – 1957), and then passes that information to the echo command. The echo command displays the text, with the results of the arithmetic (45) inserted.


#!/bin/bash
#
cd /home ; du -s * | sort -rn

This script looks at all directories in /home and reports on their size in kilobytes. The output is sorted numerically, with the largest directory at the top of the list.


$ cat /etc/password | sort | more

This command prints the contents of the /etc/password file and pipes the output to the sort command. The sort command takes the user names that begin each line of the /etc/password file, sorts them alphabetically, and pipes the output to the more command.


$ echo 'PS1="\W \!?> " ' >> ~/.bash_profile

Adds the line 'PS1="\W \!?> " ' to your ~/.bash_profile.

Make sure that you use two right-carets (>>). A single one will overwrite the file instead of appending to it.


$ ls -l $BASH_ENV
-rw-r—r-- 1 anthony  busdriver  124  Aug 10 01:50 /Users/anthony/.bashrc

In this example, you wanted to see the location of your bash environment file, and then check its size and when it was last changed.


$ export PATH=$PATH:/home/xyz/bin

In this example, I temporarily added the /home/xyz/bin directory to the PATH. This is useful if, during a shell session, you find yourself wanting to run a bunch of commands from a directory that is not normally in your PATH. This temporary addition saves you from typing the full or relative path each time you want to run a command.


$ jobs
[1]  Stopped (tty output)  vi /tmp/myfile
[2]  Running        find /usr -print > /tmp/allusrfiles &
[3]  Running        nroff -man /usr/man2/* >/tmp/man2 &
[4]- Running        nroff -man /usr/man3/* >/tmp/man3 &
[5]+ Stopped        nroff -man /usr/man4/* >/tmp/man4

To check which commands you have running in the background, use the jobs command. To see the process ID for the background job, add an -l option to the jobs command. If you type ps, you can use the process ID to figure out which command is associated with a particular background command.

Continuing with the example shown, you can bring any of the commands on the jobs list into the foreground. For example, if you are ready to edit myfile again, you can type:

$ fg %1

As a result, the vi command opens again, with all the text as it was when you stopped the vi job.


$ ps ax | less
$ kill [pid]

replace [pid] with a number

Use a virtual terminal. After you log in, you can look for the X process (ps ax | less)
and kill it (kill pid, where pid is the X process).


$ xlsfonts | wc -l

Xlsfonts outputs the names of all the fonts installed on your computer and wc -l counts how many were outputed.


$ ls -ld test
drwxr-xr-x  2 chris  sales   1024  Jan 24 12:17 test

# (excerpt from man ls)
# -d  If an argument is a directory, list only its name (not its
#     contents); often used with -l to get the status of a directory.

$ mount -t vfat /dev/fd0 /mnt/floppy
$ dd if=/dev/hda2 of=/mnt/floppy/bootsect.lnx bs=512 count=1
$ umount /mnt/floppy

Mounts a FAT formatted floppy, copies the first sector off the second partition, then unmounts the floppy.


Who am I?

  • user name
  • group name
  • user ID
  • group ID

To find out information about your identity, use the id command as follows:

$ id
uid=109(sameh) gid=102(devs) groups=102(devs),9(bigpeople),3(teachers)

These tell you sameh’s permissions.

You can see information about your current login session by using the who command. In the following example, the

  • -i option tells the who command to print the login time
  • -m says to print information about the current user
  • -H asks that a header be printed
$ who -imH
                    USER  LINE    LOGIN-TIME     IDLE  FROM
wwwsvr.gxsam11.net!sameh  tty1    Jun 18 20:57     .

FROM can display the name of the remote computer which sameh logged in from — if it was a different system. IDLE time shows the user is currently working in the shell since a dot is displayed.


Subprocess

Every time you open a shell (interactive or non-interactive) or enter a command you are starting a subprocess.

Some shell things are known to subprocesses. By shell things I am referring to shell variables (built-in or otherwise), aliases, and options.

The phrase “environment variable” refers to a special class of shell things that are available to the login shell and all subprocesses. Some (Not all) built in shell variables are environment variables. There are three ways things can be environment variables:

  • They just are environment variables.
  • They are exported with the export command.
  • They are placed in the .bashrc file (a.k.a. environment file)

An environment variable does not have to be a variable. It could be an option or an alias.

Although environment variables are guaranteed to be known to subprocesses not all shell things are known to subprocesses.

The built-in variables HOME, MAIL, PATH, and PWD are environment variables.

For automatic exporting: set -a or set -o allexport.

$ export vars

Variable names separated by blanks. Can combine assignment and exportation:

$ export var1=val1 var2=val2 var3=val3

You can define variables to be for a particular subprocess (command) only, like this:

$ name=val command

This variable assignment won’t propagate beyond this command line. This “command” includes options and arguments.

$ export
# or
$ export -p

Displays all your environment variables and their values.

$ declare

Also, displays your environment variables. Actually, declare show you more stuff than export.

Here are some variables which are not built-in — but are commonly made into environment variables:

COLUMNS The number of columns of your display
EDITOR File path of your preferred editor
LINES number of lines in your display
SHELL File path for the shell which commands will use — if they need to spawn a subshell. Compare this with BASH.
TERM Your terminal’s type

Positional Parameters

This and some other content on this page is copied or derived from Advanced Bash-Scripting Guide by Mendel Cooper. This document may only be distributed subject to the terms and conditions set forth in the Open Publication License (version 1.0 or later), http://www.opencontent.org/openpub/. See the book to learn of all legal issues related to this intellectual property.

The most important special, built-in variables are called positional parameters. These hold the command-line arguments to scripts when they are invoked. Positional parameters have the names 1, 2, 3, etc., meaning that their values are denoted by $1, $2, $3, etc. There is also a positional parameter 0, whose value is the name of the script (i.e., the command typed in to invoke it).

Two special variables contain all of the positional parameters (except positional parameter 0): * and @. The difference between them is subtle but important, and it’s apparent only when they are within double quotes.

"$*" is a single string that consists of all of the positional parameters, separated by the first character in the environment variable IFS (internal field separator), which is a space, TAB, and NEWLINE by default. On the other hand, "$@" is equal to "$1" "$2"... "$N", where N is the number of positional parameters. That is, it’s equal to N separate double-quoted strings, which are separated by spaces. If there are no positional parameters, "$@" expands to nothing. We’ll explore the ramifications of this difference in a little while.

The variable # holds the number of positional parameters (as a character string). All of these variables are “read-only,” meaning that you can’t assign new values to them within scripts.

For example, assume that you have the following simple shell script:

echo "alice: $@"
echo "$0: $1 $2 $3 $4"
echo "$# arguments"

Assume further that the script is called alice. Then if you type alice in wonderland, you will see the following output:

alice: in wonderland
alice: in wonderland
2 arguments

In this case, $3 and $4 are unset, which means that the shell will substitute the empty (or null) string for them. [3]

[3] Unless the option nounset is turned on, in which case the shell will return an error message.

Shell functions use positional parameters and special variables like * and # in exactly the same way as shell scripts do. If you wanted to define alice as a function, you could put the following in your .bash_profile or environment file:

function alice
{
    echo "alice: $*"
    echo "$0: $1 $2 $3 $4"
    echo "$# arguments"
}

You will get the same result if you type alice in wonderland.

Typically, several shell functions are defined within a single shell script. Therefore each function will need to handle its own arguments, which in turn means that each function needs to keep track of positional parameters separately. Sure enough, each function has its own copies of these variables (even though functions don’t run in their own subshells, as scripts do); we say that such variables are local to the function.

However, other variables defined within functions are not local (they are global), meaning that their values are known throughout the entire shell script. For example, assume that you have a shell script called ascript that contains this:

function afunc
{
  echo in function: $0 $1 $2
  var1="in function"
  echo var1: $var1
}

var1="outside function"
echo var1: $var1
echo $0: $1 $2
afunc funcarg1 funcarg2
echo var1: $var1
echo $0: $1 $2

If you invoke this script by typing ascript arg1 arg2, you will see this output:

var1: outside function
ascript: arg1 arg2
in function: ascript funcarg1 funcarg2
var1: in function
var1: in function
ascript: arg1 arg2

In other words, the function afunc changes the value of the variable var1 from “outside function” to “in function,” and that change is known outside the function, while $1 and $2 have different values in the function and the main script. Notice that $0 doesn’t change because the function executes in the environment of the shell script and $0 takes the name of the script. Figure 4.2 shows the scope of each variable graphically.

Figure 4.2. Functions have their own positional parameters

Local Variables in Functions

A local statement inside a function definition makes the variables involved all become local to that function. The ability to define variables that are local to “subprogram” units (procedures, functions, subroutines, etc.) is necessary for writing large programs, because it helps keep subprograms independent of the main program and of each other.

Here is the function from our last example with the variable var1 made local:

function afunc
{
  local var1
  echo in function: $0 $1 $2

  var1="in function"
  echo var1: $var1

}
var1="outside function"
echo var1: $var1
echo $0: $1 $2
afunc funcarg1 funcarg2
echo var1: $var1
echo $0: $1 $2

Now the result of running ascript arg1 arg2 is:

var1: outside function
ascript: arg1 arg2
in function: ascript funcarg1 funcarg2
var1: in function
var1: outside function
ascript: arg1 arg2

Note that afunc now has its own, local copy of var1, although the original var1 would still be used by any other functions that ascript invokes.

Notice that (although $1 and $2 are independent of their script counterparts) $0 refers to the same thing in and out of the function. $0 always refers to the script name.

By default $1 and $2 are local when inside a function. Whereas, $0 is neither local or global — it’s just the name of the nearest script from above.

Quoting with $@ and $*

Now that we have this background, let’s take a closer look at "$@" and "$*". These variables are two of the shell’s greatest idiosyncracies, so we’ll discuss some of the most common sources of confusion.

  • Why are the elements of "$*" separated by the first character of IFS instead of just spaces? To give you output flexibility. As a simple example, let’s say you want to print a list of positional parameters separated by commas. This script would do it:

    IFS=,
    echo "$*"
    

    Changing IFS in a script is risky, but it’s probably OK as long as nothing else in the script depends on it. If this script were called arglist, then the command arglist alice dormouse hatter would produce the output alice,dormouse,hatter. Chapter 5 and Chapter 10 contain other examples of changing IFS.

  • Why does "$@" act like N separate double-quoted strings? To allow you to use them again as separate values. For example, say you want to call a function within your script with the same list of positional parameters, like this:

    function countargs
    {
        echo "$# args."
    }
    

    Assume your script is called with the same arguments as arglist above. Then if it contains the command countargs "$*", the function will print 1 args. But if the command is countargs "$@", the function will print 3 args.


    File System

    $ ls -al CodeIgniter
    total 136
    drwxr-xr-x  17 samehlabib  staff    578 Nov 28 18:40 .
    drwxr-xr-x+ 44 samehlabib  staff   1496 Nov 26 19:10 ..
    drwxr-xr-x  11 samehlabib  staff    374 Nov 28 19:49 .git
    -rw-r--r--   1 samehlabib  staff    304 Nov 24 11:12 .gitignore
    -rw-r--r--   1 samehlabib  staff    771 Nov 28 18:40 .travis.yml
    -rw-r--r--   1 samehlabib  staff   1089 Nov 24 11:12 DCO.txt
    drwxr-xr-x  16 samehlabib  staff    544 Nov 24 11:12 application
    -rw-r--r--   1 samehlabib  staff    110 Nov 24 11:12 composer.json
    -rw-r--r--   1 samehlabib  staff   6611 Nov 24 11:12 contributing.md
    

    The plus (+) indicates a directory has subdirectories.

    Dot (.) by itself refers to the current directory.

    Dot dot (..) by itself refers to the parent directory.

    Dot before a file or directory name means it’s considered to be hidden.

    d in front of rwxr-xr-x means the listing is for a directory.

    A number (like 44 above) is the number of files.

    A number (like 578 above) is the number of bytes associated with the file (in the case of a directory it’s the number of bytes that make up the directory definition file.)

    samehlabib is the owner and he belongs to the staff group.

    The date and time refers to when a file was last modified.

    Bash’s cd in the form cd -, changes to previous directory. cd without an argument takes you to your home directory — just like cd ~.

    Create a Symlink

    $ sudo ln -s "/Apps/Sublime Text 2.app/bin/subl" /bin/subl
    # ln -s <destination> <linkname>
    # If linkname is w/o path then symlink is created in current directory
    # sudo command -- need root privilege to create things in /bin/subl
    # Quotes were used to hold together first argument because spaces.
    # There's also symlink command.
    

    Tilde (~) home-directory

    ~james/user/directory
    ~/my/user/directory
    

    A file path which uses a tilde is considered an absolute-file-path.

    pwd (the command)

    Print the name of the current working directory.

    mkdir

    Create a directory.

    chmod

    Change the permission on a file or directory.


    Commands (various aspects of using them)

    Options can have arguments.

    lp -d lp1 -h file
    

    The process of matching expressions containing wildcards to filenames is called wildcard expansion or globbing.

    Tilde expansion is another form of command line processing.

    Brace expansion (see next heading) can also be used for filepath expansion.

    Commands only see results of wildcard expansion — They see an argument list.

    wildcard what it matches
    ? any single character
    * any string — including an empty one
    [set] any one character in this set
    [!set] any one character not in this set

    You can specify a set using a dash-separated range of letters or numbers.

    the set what it matches
    [abc] a, b, or c
    [.,;] a period, comma, or semicolon
    [-_] a dash or an underscore
    [!0-9] any non-digit character
    [0-9!] any digits or an exclamation point
    [a-zA-Z] any lowercase or uppercase letter
    [a-zA-Z0-9_-] any letter, digit, underscore, or dash

    To match ! using set notation, place it after at least one other character, or precede it with a backslash — [\!].

    If no filenames are found which match then the shell will return the pattern itself.

    You can use wildcards as part of a pathname.

    ls /www*/[ikr]?
    

    Brace Expansion

    Whereas with filepath wildcard expansion the expression will expand to files and directories which exist, brace expansion expands to an arbitrary string.

    $ echo cam{co,per,d}s
    camcos campers camds
    

    It is also possible to nest braces, like vd{m{w,q,po},lpy}dv. This would give the same result as vd{mw,mq,mpo,lpy}dv.

    You can combine both types of expansion:

    ls *.{htm,html,php}
    # is equivalent to:  ls *.htm *.html *.php
    # I'm assuming ls can take multiple arguments.
    # If not, then it behaves as if ls can take multiple arguments
    

    Command Substitution

    One thing you can use command substitution for is to assign a value to a variable. It allows you to assign the standard output of a command to a named variable.

    Using backquotes (`) is one way to do this. However that’s not the best syntax for doing this. See example:

    $ LsOutput=`ls -al`
    # Store command output in a variable using backquotes.
    # The other way to do this is:
    $ LsOutput=$(ls -al)
    

    (for the line which has the $(command)) the command inside the parentheses is run, and anything the command writes to standard output is returned. These constructs can be nested.

    Besides using command substitution to assign a value to a variable name, you can use it to provide arguments to another command:

    vi $(grep -l 'needle' filename*)
    

    This opens up (into vi) every file (in the current directory whose name starts with filename) which contains needle.

    The -l option to grep causes it only to print names of files that contain matches.

    Command substitution (just like variable and tilde expansion) occurs inside double quotes.

    Arithmetic Expressions Expansion

    There are two forms you can use to expand an arithmetic expression and pass it to the shell: $[expression] or $((expression)). Here is an example:

    $ echo "There were $[2 + 6] monkeys at the zoo."
    There were 8 monkeys at the zoo.
    

    Input/Output

    UNIX file I/O takes the form of arbitrarily long sequence of characters (bytes)—as opposed to blocks, records, or card images. Everything on the system that produces or accepts data is treated as a file; this includes hardware devices like disk drives and terminals.

    Standard I/O

    Every UNIX program that you run has these standard I/O streams:

    • standard input — defaults to keyboard
    • standard output — defaults to terminal display
    • standard error — defaults to terminal display

    Standard I/O streams can be redirected by the shell to a file or another program.

    Even a PHP script made for the command line will have these streams built in by default.

    Generally programs/commands take input from a file whose name is an argument; However, if you omit this filepath argument, the program will expect its input from the keyboard. Here is what you can expect if you omit the filepath:

    • If the program doesn’t expect input then it may just produce some output then return the shell prompt. Otherwise ..
    • You will be at the start of a new line (which may or may not have some type of prompt specific to this program.)
    • You are free to type on this line
    • You can enter a newline character (Enter key) to mark the end of a line and start to start typing on the next line.
    • Some character or string you type will mark the end of file (usually the ASCII character for EOF which is produced by the Ctrl-d a.k.a. ^D key) and cause the program to process your input. Some programs consider the newline character to be the mark of the end of file — hence the enter key will serve this goal — thus preventing you from entering a multiple line file (as your input.)
    • Program produces program output and error output.
    • Program may expect input again.
    • If the program took more input then it may produce more program output and error output.
    • Your shell prompt comes back when the program decides this is its appropriate response at that point in time.

     

    Piping

    Redirect all output of a command into the standard input of another command.

    ps -ax | tee somefile.ascii | more
    cat /tmp/otherfile | sort | more
    

    Redirection

    < Direct the contents of a file to the command — assuming the command expects input from the keyboard.
    > Direct the output of a command to a file, deleting the existing file — assuming the command was going to output to the screen.
    >> Direct the output of a command to a file, adding the output to the end of the existing file — assuming the command was going to output to the screen.

    example:

    $ echo "My next colonoscopy is on $(nextappointment)" >> ~/appointments
    

    Redirection can be combined:

    command < filename1 > filename2
    

    The One Line Script using (;) Semicolon

    $ cd ; pwd ; ls
    

    Use a semicolon (;) to run commands in sequence.


    Utilities (UNIX Programs & Bash Built-ins)

    Built-in

    For information on individual bash built-in commands see the Bash Man Page or use the --help option with the command.

    The following is a comma separated list of built-ins: bash, :, ., [, alias, bg, bind, break, builtin, cd, command, compgen, complete, continue, declare, dirs, disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help, history, jobs, kill, let, local, logout, popd, printf, pushd, pwd, read, readonly, return, set, shift, shopt, source, suspend, test, times, trap, type, typeset, ulimit, umask, unalias, unset, wait

    Not Built-in

    Command What it’s for
    id To find out information about your identity.
    nslookup query Internet name servers interactively
    ifconfig Displays the status of the currently active network interfaces. Also, it serves as a command-line utility for configuring network interfaces.
    netstat show network status
    who Display who is logged in.
    route Manually manipulate the routing tables.
    uname Print operating system name.
    hostname Set or print name of current host system.
    domainname Set or print name of current YP/NIS domain.
    dmesg Display the system message buffer.
    mknod Make device special file.
    passwd modify a user’s password
    xset user preference utility for X
    startx initialize an X session
    less Types the contents of a file onto the screen one page at a time.
    tr The tr utility copies the standard input to the standard output with substitution or deletion of selected characters.
    file determine file type
    cat concatenate and print files
    head display first lines of a file
    tail output the last part of files
    sort sort lines of text files
    grep grep, egrep, fgrep – print lines matching a pattern
    cmp compare two files byte by byte
    diff compare files line by line
    sed stream editor
    awk pattern-directed scanning and processing language
    pr The pr utility is a printing and pagination filter for text files.
    cut cut out selected portions of each line of a file
    script make typescript of terminal session
    magic The file command’s magic pattern file.
    cd change working directory
    ls list directory contents
    basename basename, dirname — return filename or directory portion of pathname
    stat readlink, stat — display file status
    chmod change file modes or Access Control List
    touch change file access and modification times
    df display free disk space
    du estimate file space usage
    ln make links between files
    mount mount file systems
    mkdir make directories
    rmdir remove directories
    rm remove directory entries
    mv move files
    cp copy files
    find walk a file hierarchy
    pwd return working directory name
    locate find filenames quickly
    echo write arguments to the standard output
    tee The tee utility copies standard input to standard output, making a copy in zero or more files. The output is unbuffered.
    scp secure copy (remote file copy program)
    fdisk DOS partition maintenance program
    umount unmount filesystems
    shutdown close down the system at a given time
    resize set TERMCAP and terminal settings to current xterm window size
    dd convert and copy a file
    top display and update sorted information about processe
    ps process status
    nice execute a utility with an altered scheduling priority
    killall killall
    fuser list process IDs of all processes that have one or more files open
    at at, batch, atq, atrm — queue, examine, or delete jobs for later execution
    cron daemon to execute scheduled commands
    gzip compress or expand files
    tar manipulate tape archives
    date display or set date and time
    ntpdate set the date and time via NTP
    cal displays a calendar and the date of easter
    xargs construct argument list(s) and execute utility
    which locate a program file in the user’s path
    man format and display the on-line manual pages
    apropos search the whatis database for strings
    screen screen manager with VT100/ANSI terminal emulation
    clear clear the terminal screen
    tput initialize a terminal or query terminfo database
    terminfo terminal capability data base
    lpr print files
    lp print files
    pr print files
    sleep suspend execution for an interval of time
    su substitute user identity
    sudo execute a command as another user
    chpass add or change user database information
    talk talk to another user
    finger user information lookup program

Posted in UNIX Command Line | 1 Comment

Bash Vi Command Line Editing


related: Using the Bash CLI


Excerpts paraphrased from the GNU Bourne-Again SHell manual

Command line editing is provided by the Readline library, which is used by several different programs, including Bash. By default, the line editing commands are similar to those of Emacs. A vi-style line editing interface is also available. Line editing can be enabled at any time using the -o emacs or -o vi options to the set builtin command, or disabled using the +o emacs or +o vi options to set.

While the Readline library does not have a full set of vi editing functions, it does contain enough to allow simple editing of the line. The Readline vi mode behaves as specified in the POSIX standard.

When you enter a line in vi mode, you are already placed in ‘insertion’ mode, as if you had typed an 'i'. Pressing ESC switches you into ‘command’ mode, where you can edit the text of the line with the standard vi movement keys, move to previous history lines with 'k' and subsequent lines with 'j', and so forth.


In this blog post there may be some confusion. Mostly I’ll be talking about command line editing; Occasionally I may talk about using a command line editor. You can always fire up the vi text editor by issuing its command. The crux of this post is about using programs in the mode where they’ll be flavored with vi. For example: You may be running Git at the terminal. Some of the Git features ask you to edit text. If you want this done in vi-style then keep reading.

$ set -o vi

$ set -o emacs
# Undo set -o vi

Tells the Readline library to use vi-style. This will affect command editing, command history and programs you run in the terminal which do use the Readline library.


$ export EDITOR=vi VISUAL=vi

Signals to programs which do NOT use the Readline library that vi-style is your preferrence — by setting environment variables.

export — specifies that those two variables are to have global scope. In other words they are valid for any sub-shell or script you may run from the current shell. Child processes cannot export variables back to the parent processes that spawned them.


Both emacs- and vi-modes introduce the potential for clashes with control keys set up by the UNIX terminal interface. These control keys override their functions in the editing modes.


Changing command line editing to vi-style by using the set command or by assigning the variables EDITOR/VISUAL a value of vi (and export) will only affect the current shell and its child processes.


If you start bash with the -noediting option there will be no command line editing.


All of bash’s command history facilities depend on a list that records commands as you type them into the shell. Whenever you log in or start another interactive shell, bash reads an initial history list from the file .bash_history in your home directory. From that point on, every bash interactive session maintains its own list of commands. When you exit from a shell, it saves the list in .bash_history. You can call this file whatever you like by setting the environment variable HISTFILE.


Vi-mode essentially creates a one-line editing window into the history file.

Input mode — is for typing commands (as in normal bash use)

Control mode — is for moving around the command line and the history file.


Here’s more about input mode:

When you are in input mode, you can type commands in and hit RETURN to run them. In addition, you have minimal editing capabilities via control characters, which are summarized shown here.

Command Description
BACKSPACE Delete previous character
DEL Delete current character
CTRL-w Erase previous word (i.e., erase until a blank)
CTRL-v Quote the next character (meaning echo the next char I type)
ESC Enter control mode

When you use CTRL-v you are telling bash: “the next thing I enter at the keyboard is a single character which you will insert.”


Movement Commands (in Command Mode)

non-blank word — is any sequence of non-blank characters

word — is any sequence of only alphanumeric characters (letters and digits) plus the underscore (_), or any sequence of only non-alphanumeric characters

Command Description
h Move left one character
l Move right one character
w Move right one word
b Move left one word
W Move right to beginning of next non-blank word
B Move left to beginning of preceding non-blank word
e Move to end of current word
E Move to end of current non-blank word
0 Move to beginning of line
^ Move to first non-blank character in line
$ Move to end of line

All of these commands except the last three can be preceded by a number that acts as a repeat count. Whenever you type a number for the repeat count, the number replaces the command prompt for the duration of the repeat command. If your keyboard has cursor motion keys (“arrow” keys), you can use the left and right arrows to move between characters instead of the h and l keys. Repeat counts will work with the cursor keys as well.


Commands for Entering Input Mode

Command Description
i Text inserted before current character (insert)
a Text inserted after current character (append)
I Text inserted at beginning of line
A Text inserted at end of line
R Text overwrites existing text

Deletion (in Command Mode)

Command Description
dh Delete one character backwards
dl Delete one character forwards
db Delete one word backwards
dw Delete one word forwards
dB Delete one non-blank word backwards
dW Delete one non-blank word forwards
d$ Delete to end of line
d0 Delete to beginning of line

These commands have a few variations and abbreviations. If you use a c instead of d, you will enter input mode after it does the deletion. You can supply a numeric repeat count either before or after the d (or c). This table lists the available abbreviations.

Command Description
D Equivalent to d$ (delete to end of line)
dd Equivalent to 0d$ (delete entire line)
C Equivalent to c$ (delete to end of line, enter input mode)
cc Equivalent to 0c$ (delete entire line, enter input mode)
X Equivalent to dl (delete character backwards)
x Equivalent to dh (delete character forwards)

un-delete commands — vi-mode maintains a delete buffer that stores all of the modifications to text on the current line only (note that this is different from the full vi editor). The command u undoes previous text modifications. A related command is . (dot), which repeats the last text modification command.

There is also a way to save text in the delete buffer without having to delete it in the first place: just type in a delete command but use y ("yank") instead of d. This allows you to retrieve the yanked text as many times as you like later on. The commands to retrieve yanked text are p, which inserts the text to the right of the cursor, and P, which inserts it to the left of the cursor.


History

Command Description
k or - Move backward one line
j or + Move forward one line
G Move to line given by repeat count
/string Search backward for string
?string Search forward for string
n Repeat search in same direction as previous
N Repeat search in opposite direction of previous

(more) Movement

Command Description
fx Move right to next occurrence of x
Fx Move left to previous occurrence of x
tx Move right to next occurrence of x, then back one space
Tx Move left to previous occurrence of x, then forward one space
; Redo last character-finding command
, Redo last character-finding command in opposite direction

One final command rounds out the vi control mode commands for getting around on the current line: you can use the pipe character (|) to move to a specific column, whose number is given by a numeric prefix argument. Column counts start at 1; count only your input, not the space taken up by the prompt string. The default repeat count is 1, of course, which means that typing | by itself is equivalent to 0.


Text Completion

Although not a feature of the vi editor — Text completion is available in vi-mode.

Backslash (\) is the command that tells bash to do completion. If you type in a word, hit ESC to enter control mode, and then type \, one of four things will happen:

  1. If there is nothing whose name begins with the word, the shell will beep and nothing further will happen.
  2. If there is a command name in the search path, a function name, or a filename that the string uniquely matches, the shell will type the rest of it, followed by a space in case you want to type in more command arguments. Command name completion is only attempted when the word is in a command position (e.g: at the start of a line).
  3. If there is a directory that the string uniquely matches, the shell will complete the filename, followed by a slash.
  4. If there is more than one way to complete the name, the shell will complete out to the longest common prefix among the available choices. Commands in the search path and functions take precedence over filenames.

A related command is *. It behaves similarly to ESC-\, but if there is more than one completion possibility (number four in the previous list), it lists all of them and allows you to type further. Thus, it resembles the * shell wildcard character.

Less useful is the command =, which does the same kind of expansion as *, but in a different way. Instead of expanding the names onto the command line, it prints them, then gives you your shell prompt back and retypes whatever was on your command line before you typed =.


Other Bash vi command mode commands:

CTRL-L — Clear the screen and redraw the current line on it.

_ — Append last word of previous command, enter input mode. A repeat count preceding _ causes the nth word in the previous command to be inserted in the current line; without the count, the last word is used. Omitting the repeat count is useful because a filename is usually the last thing on a UNIX command line, and because users often run several commands in a row on the same file.

~ — Invert (twiddle) case of current character(s). A repeat count of n preceding the ~ changes the case of the next n characters. The cursor will advance accordingly.

# — Prepend # (comment character) to the line and send it to the history file; useful for saving a command to be executed later without having to retype it.


Posted in Configure My Systems, UNIX Command Line | 1 Comment

Important Git Facts

From genjusz.wordpress.com Git — setting up a remote repository and doing an initial “push”

Keep My GitHub Fork Up-to-date

Git home

When you clone a project/repository you are copying all its files and metadata.

Staged means that you have marked a modified file in its current version to go into the repository the next time you commit. Or, the term staged could refer to a new file being marked for inclusion in the next commit.

This file will be committed in the state it was in when you staged it.

A working directory is a single checkout of one version/branch of a project/repository.

Within the working directory you’ll find the project files (for the checked-out branch) and the git metadata of the project (in .git hidden sub-directory.)

If (later-on) you switch branches you Do Not have to look somewhere else in file system to find project files for the branch you switched to. They magically appear in the same place where the previous branch files were.

Index is a term which refers to the staging area.

File state:

  • modified
  • removed
  • renamed
  • tracked/(previously committed)
  • staged

$ git config

to alter git configuration files

You can configure at these “levels”:

  • --system
  • --global
  • current repository

You can set a user name and email which is different from the global one — on a per project basis.

You can check what Git thinks a specific configuration key’s value is by typing
git config {key}

Help!

$ git help <verb>
$ git <verb> --help
$ man git-<verb>

url = https://github.com/Sameh-R-Labib/CodeIgniter-3-Source-Explained.git

You will find a “text segment” like this in .git/config within your local repository’s folder. That’s the only place where you can find something that can be construed as a git repository name.

Here, CodeIgniter-3-Source-Explained will be the repository name — if there is such a thing as a repository name.

If I was creating that remote repository (w/o GitHub UI) using SSH from my computer at home I would have done:

mkdir CodeIgniter-3-Source-Explained.git
cd CodeIgniter-3-Source-Explained.git
git init --bare

This creates a repository without a .git directory or a working directory.

My workflow: — 1. create repo on GitHub … 2. clone it to my computer using git clone or GitHub.app

$ git init

You issue the git init command at a directory you want to turn into a repo. Issuing this command creates a new subdirectory named .git which contains all of your necessary repository files. You won’t be tracking anything yet — since you haven’t made any commits.

.git

is a hidden sub-directory of the working directory of a project

The Git repository is stored in this directory.

$ git clone https://github.com/username/Straw-Hey.git
# Clones your fork of the repo into the current directory in terminal

See GitHub instructions.

$ git clone git://gxsam11.net/samehlabib/customfw.git

is an example for cloning a remote repository

creates a directory named customfw, initializes a .git directory inside it, pulls down all the data for that repository, and checks out a working copy of the latest version.

I get the feeling this will be created in the current directory.

You can have this command give the working directory a different name by specifying it as a parameter (at the end.)

Git can use a variety of transfer protocols. The previous example uses the git:// protocol, but you may also see http(s):// or user@server:/path.git, which uses the SSH transfer protocol.

When you first clone a repository, all of your files will be tracked and unmodified because you just checked them out and haven’t edited anything.

$ git status

to determine which files are in which state … also indicates which branch you are on

After clone the status will be no (tracked && modified) files.

Several branches can exist within a single repository. Every branch has a name — Each repository has at least a master (that’s its name) branch.

git status command will tell you which files are untracked.

Untracked basically means that Git sees a file you didn’t have in the previous snapshot (commit); Git won’t start including it in your commit snapshots until you explicitly tell it to do so.

git add

to begin tracking a new file

If you commit at this point, the version of the file at the time you ran git add is what will be in the historical snapshot.

The git add command takes a path name for either a file or a directory; if it’s a directory, the command adds all the files in that directory recursively.

After a previously tracked file is modified it has to be staged in order to get committed in its new (modified) form.

The git add command can be used to mark merge-conflicted files as resolved.

git mv

mark files to be moved (renamed) — within the historical snapshots for the branch

git rm [file]

mark file to be removed from index (staging area); And, the file will be deleted from your working directory.

If you simply deleted the file yourself you still need to use this command before you commit. Doing a git status will show this.

git reset HEAD [file]

to unstage

git status

Use this before you commit to see which files need to be staged.

git diff [file]

or just

git diff — compares what is in your working directory with what is in your staging area

git diff --cached — command compares your staged changes to your last commit

Generally speaking — Use diff before you commit so you can see exactly what you’ve modified in a file — or all files which are in modified status.

shows you the lines added and the lines removed (the patch)

git commit

The simplest way to commit. But, will open an editor for you to enter the commit comment.

git commit -a

automatically adds modified files; then takes snapshot — Note: it won’t add the files which are new.

git commit -a -m [message]

specifies a message to associate with this commit

You will be committing to the current branch.

git log

shows a log of all commits for the branch

Historical snapshot == A commit stored in git

You may have a class of files which you are sure you don’t want Git track. If that’s the situation then create a file which lists all the patterns for the file names of files in that class. Name that file .gitignore. See Git website for more on this.

I believe .gitignore is similar to configuration files in that there are several places in the file system you can place them — but the most local setting for a particular key will apply.

$ cat [file]

useful UNIX command for displaying a file’s content

$ cat ~/.gitconfig

example

Posted in CI Source Code Explained, Configure My Systems, Other, UNIX Command Line | 1 Comment

My New Collaborative Blogging Workflow

About $this

I met Philipp Tempel (PhilTem) on the CodeIgniter Forum.

I was asking for help writing my series explaining the CodeIgniter source code.

Phil suggested a different workflow for blogging. One that’s conducive to collaboration.

This is all new to me. So baby steps it shall be!

Related

Important Git Facts

Got GitHub (account)

URL: https://github.com/

U/N: Sameh-R-Labib

EMAIL: my Yahoo! email

Installed a Git GUI client

 

Downloaded: GitHub.app (git client)

Allows gui interaction with Git (local); and sync to GitHub (remote).

Learned some Git

Git Basics

Every time you commit, or save the state of your project in Git, it basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot.

This makes Git more like a mini filesystem

.. it’s impossible to change the contents of any file or directory without Git knowing about it.

Git allows and encourages you to have multiple local branches that can be entirely independent of each other. …

… when you push to a remote repository, you do not have to push all of your branches. You can choose to share just one …

With Git, nearly all operations are performed locally

.. Distributed SCM, .. This means that instead of doing a “checkout” of the current tip of the source code, you do a “clone” of the entire repository. … every user essentially has a full backup of the main server. …

.. centralized workflow .. Git will not allow you to push if someone has pushed since the last time you fetched

I’ll be the only one to push to the ‘blessed’ repository (my repo on GITHUB); So, that won’t be a problem. However, if I pull (to my repo on GITHUB) from a contributor then I’ll need a fresh copy locally.

Integration Manager Workflow … involves an integration manager — a single person who commits to the ‘blessed’ repository. A number of developers then clone from that repository, push to their own independent repositories, and ask the integrator to pull in their changes. This is the type of development model often seen with open source or GitHub repositories.

Dictator and Lieutenants Workflow — For more massive projects … merge in all changes related to that subsystem. Another integrator (the ‘dictator’) can pull changes from only his/her lieutenants and then push to the ‘blessed’ repository that everyone then clones from again.

.. This means that if you have a commit ID, you can be assured not only that your project is exactly the same as when it was committed, …

.. Git has something called the “staging area” or “index”. This is an intermediate area where commits can be formatted and reviewed before completing the commit. … git add instead of git commit.

.. it’s possible to quickly stage some of your files and commit them without committing all of the other modified files in your working directory …

Installed Git (the git program)

Downloaded from: http://git-scm.com/download/mac

Mounted the disc image.

Ran the install. Which installed git to:

/usr/local/git

Restarted my computer.

I ran a shell script which came with download — sets up the Git PATH for non-terminal programs.

Logged out of (then into) my computer account.

Saved the download to Dropbox.

Configured Git

Opened a shell and entered:

$ git config --global user.name "Sameh R. Labib"

$ git config --global user.email "my Yahoo! email"

Git and GitHub emails match.

To tell git that I don’t want to type my username and password every time I talk to a remote server I had to install the osxkeychain credential helper — and tell git to use it.

sameh-labibs-imac:~ samehlabib$ curl -s -O http://github-media-downloads.s3.amazonaws.com/osx/git-credential-osxkeychain
sameh-labibs-imac:~ samehlabib$ chmod u+x git-credential-osxkeychain
sameh-labibs-imac:~ samehlabib$ which git
/usr/local/git/bin/git
sameh-labibs-imac:~ samehlabib$ sudo mv git-credential-osxkeychain /usr/local/git/bin/
Password:
sameh-labibs-imac:~ samehlabib$ git config --global credential.helper osxkeychain
sameh-labibs-imac:~ samehlabib$ 

Manual says: The next time you clone an HTTPS URL that requires a password you will be prompted for your username and password, and to grant access to the OSX keychain. After you’ve done this, the username and password are stored in your keychain and you won’t be required to type them in to git again.

I assume that my git credentials are the same as for my samehlabib computer account.

Created 1st GitHub Repository

sams-first-repo-just-created

https://github.com/Sameh-R-Labib/CodeIgniter-3-Source-Explained

This was accomplished by clicking the “Create a New Repo” button in my user bar’s right side.

I chose to have GitHub initialize the repo with a README. This will allow me to git clone the repository immediately.

Git Workflow

Integration Manager Workflow is the git workflow I’ll use.

I’m the integration manager.

Sameh-R-Labib/CodeIgniter-3-Source-Explained on GitHub is the ‘blessed’ repo.

Changes, made by contributors, to the ‘blessed’ repo (master branch on GitHub) happen when I pull them.

Pull request workflow:

  1. Contributor asks me to pull
  2. I examen his branch (on GitHub) of my repo.
  3. (assume I approve)
  4. I pull his branch into the master (on GitHub)
  5. On my computer I’ll merge local branches with the master

Cloning to Local

GitHub for Mac — Help & Keyboard Shortcuts.

I used Preferences in GitHub.app to log into GitHub.

Now I have:

github-app-logged-into-gh

I did Clone to Computer (in GitHub.app) for my GITHUB repo branch.

The location for my local repo is now:

/Users/samehlabib/CodeIgniter-3-Source-Explained
Posted in CI Source Code Explained, Configure My Systems, UNIX Command Line | Leave a comment

URI.php CI Explained

I’ve stopped work on this series to start a new one for CI 3.

URI.php is a core CodeIgniter class. This post is part of a series which explains the CodeIgniter (CI) source code. This post explains the CI 2.1.3 URI.php.

./system/core/URI.php

It’s assumed you know what URI means in the context of CodeIgniter. If you don’t then read the user guide (especially this UG page.)

The Script’s Boilerplate

<?php  if ( ! defined('BASEPATH')) exit('No direct script access allowed');
/**
 * CodeIgniter
 *
 * An open source application development framework for PHP 5.1.6 or newer
 *
 * @package		CodeIgniter
 * @author		ExpressionEngine Dev Team
 * @copyright	Copyright (c) 2008 - 2011, EllisLab, Inc.
 * @license		http://codeigniter.com/user_guide/license.html
 * @link		http://codeigniter.com
 * @since		Version 1.0
 * @filesource
 */

// ------------------------------------------------------------------------

/**
 * URI Class
 *
 * Parses URIs and determines routing
 *
 * @package		CodeIgniter
 * @subpackage	Libraries
 * @category	URI
 * @author		ExpressionEngine Dev Team
 * @link		http://codeigniter.com/user_guide/libraries/uri.html
 */
class CI_URI {

Declaration of Properties

/**
 * List of cached uri segments
 *
 * @var array
 * @access public
 */
var	$keyval			= array();
/**
 * Current uri string
 *
 * @var string
 * @access public
 */
var $uri_string;
/**
 * List of uri segments
 *
 * @var array
 * @access public
 */
var $segments		= array();
/**
 * Re-indexed list of uri segments
 * Starts at 1 instead of 0
 *
 * @var array
 * @access public
 */
var $rsegments		= array();

Code: __construct()

/**
 * Constructor
 *
 * Simply globalizes the $RTR object.  The front
 * loads the Router class early on so it's not available
 * normally as other classes are.
 *
 * @access	public
 */
function __construct()
{
	$this->config =& load_class('Config', 'core');
	log_message('debug', "URI Class Initialized");
}

I have no idea what this comment means:

 * Simply globalizes the $RTR object.  The front
 * loads the Router class early on so it's not available
 * normally as other classes are.

What I do know:

The constructor creates a property $config. Its value is (by-reference) the configuration object.

And, it does: log_message('debug', "URI Class Initialized")

Code: _fetch_uri_string()

Establishes the current URI string and stores it in $uri_string

/**
 * Get the URI String
 *
 * @access	private
 * @return	string
 */
function _fetch_uri_string()
{
	if (strtoupper($this->config->item('uri_protocol')) == 'AUTO')
	{
		// Is the request coming from the command line?
		if (php_sapi_name() == 'cli' or defined('STDIN'))
		{
			$this->_set_uri_string($this->_parse_cli_args());
			return;
		}

		// Let's try the REQUEST_URI first, this will work in most situations
		if ($uri = $this->_detect_uri())
		{
			$this->_set_uri_string($uri);
			return;
		}

		// Is there a PATH_INFO variable?
		// Note: some servers seem to have trouble with getenv() so we'll test it two ways
		$path = (isset($_SERVER['PATH_INFO'])) ? $_SERVER['PATH_INFO'] : @getenv('PATH_INFO');
		if (trim($path, '/') != '' && $path != "/".SELF)
		{
			$this->_set_uri_string($path);
			return;
		}

		// No PATH_INFO?... What about QUERY_STRING?
		$path =  (isset($_SERVER['QUERY_STRING'])) ? $_SERVER['QUERY_STRING'] : @getenv('QUERY_STRING');
		if (trim($path, '/') != '')
		{
			$this->_set_uri_string($path);
			return;
		}

		// As a last ditch effort lets try using the $_GET array
		if (is_array($_GET) && count($_GET) == 1 && trim(key($_GET), '/') != '')
		{
			$this->_set_uri_string(key($_GET));
			return;
		}

		// We've exhausted all our options...
		$this->uri_string = '';
		return;
	}

	$uri = strtoupper($this->config->item('uri_protocol'));

	if ($uri == 'REQUEST_URI')
	{
		$this->_set_uri_string($this->_detect_uri());
		return;
	}
	elseif ($uri == 'CLI')
	{
		$this->_set_uri_string($this->_parse_cli_args());
		return;
	}

	$path = (isset($_SERVER[$uri])) ? $_SERVER[$uri] : @getenv($uri);
	$this->_set_uri_string($path);
}

let us break _fetch_uri_string() down

if (strtoupper($this->config->item('uri_protocol')) == 'AUTO')
{

See ./application/config/config.php configuration file to understand what the individual settings represent. From there we find:

/*
|--------------------------------------------------------------------------
| URI PROTOCOL
|--------------------------------------------------------------------------
|
| This item determines which server global should be used to retrieve the
| URI string.  The default setting of 'AUTO' works for most servers.
| If your links do not seem to work, try one of the other delicious flavors:
|
| 'AUTO'			Default - auto detects
| 'PATH_INFO'		Uses the PATH_INFO
| 'QUERY_STRING'	Uses the QUERY_STRING
| 'REQUEST_URI'		Uses the REQUEST_URI
| 'ORIG_PATH_INFO'	Uses the ORIG_PATH_INFO
|
*/
$config['uri_protocol']	= 'AUTO';

back to _fetch_uri_string()

If uri_protocol is 'AUTO' then:

Execute five if statements and two regular statements.

Here’s the first if statement.

// Is the request coming from the command line?
if (php_sapi_name() == 'cli' or defined('STDIN'))
{
	$this->_set_uri_string($this->_parse_cli_args());
	return;
}

If CI is being used from the command line then use _parse_cli_args to feed _set_uri_string… and we are done. This will establish the property uri_string.

Here’s the second if statement.

// Let's try the REQUEST_URI first, this will work in most situations
if ($uri = $this->_detect_uri())
{
	$this->_set_uri_string($uri);
	return;
}

Try the _detect_uri method. If it works then use its return value as an argument to _set_uri_string … and we are done. The property uri_string will have been set. Which means we have established the current uri string.

Here’s the third if statement.

// Is there a PATH_INFO variable?
// Note: some servers seem to have trouble with getenv() so we'll test it two ways
$path = (isset($_SERVER['PATH_INFO'])) ? $_SERVER['PATH_INFO'] : @getenv('PATH_INFO');
if (trim($path, '/') != '' && $path != "/".SELF)
{
	$this->_set_uri_string($path);
	return;
}

This is some other way of trying to established the current uri string.

Here’s the fourth if statement.

// No PATH_INFO?... What about QUERY_STRING?
$path =  (isset($_SERVER['QUERY_STRING'])) ? $_SERVER['QUERY_STRING'] : @getenv('QUERY_STRING');
if (trim($path, '/') != '')
{
	$this->_set_uri_string($path);
	return;
}

This is some other way of trying to established the current uri string.

Here’s the fifth if statement.

// As a last ditch effort lets try using the $_GET array
if (is_array($_GET) && count($_GET) == 1 && trim(key($_GET), '/') != '')
{
	$this->_set_uri_string(key($_GET));
	return;
}

// We've exhausted all our options...
$this->uri_string = '';
return;

Try one last thing before giving up and setting the current uri string to an empty string.

END OF: If uri_protocol is 'AUTO'.

$uri = strtoupper($this->config->item('uri_protocol'));

if ($uri == 'REQUEST_URI')
{
	$this->_set_uri_string($this->_detect_uri());
	return;
}
elseif ($uri == 'CLI')
{
	$this->_set_uri_string($this->_parse_cli_args());
	return;
}

$path = (isset($_SERVER[$uri])) ? $_SERVER[$uri] : @getenv($uri);
$this->_set_uri_string($path);

Just more things to try in order to set the current uri string

Code: _set_uri_string()

This is a helper which sets the current uri string ($this->uri_string) when given (as an argument) the path resulting from one of some other helpers — These “other helpers” have specialized approaches to extracting the path from the system.

/**
 * Set the URI String
 *
 * @access	public
 * @param 	string
 * @return	string
 */
function _set_uri_string($str)
{
	// Filter out control characters
	$str = remove_invisible_characters($str, FALSE);

	// If the URI contains only a slash we'll kill it
	$this->uri_string = ($str == '/') ? '' : $str;
}

To better understand how the URI class derives the URI from the system we need to understand what form $str is in. To accomplish this understanding we must explain how $str becomes $uri_string.

Based on analysis (which I did in my head) I’ve concluded:

Both $str and $uri_string essentially have the same form.

So, the magic happens in those “other helpers”.

Code: _detect_uri()

/**
 * Detects the URI
 *
 * This function will detect the URI automatically and fix the query string
 * if necessary.
 *
 * @access	private
 * @return	string
 */
private function _detect_uri()
{
	if ( ! isset($_SERVER['REQUEST_URI']) OR ! isset($_SERVER['SCRIPT_NAME']))
	{
		return '';
	}

	$uri = $_SERVER['REQUEST_URI'];
	if (strpos($uri, $_SERVER['SCRIPT_NAME']) === 0)
	{
		$uri = substr($uri, strlen($_SERVER['SCRIPT_NAME']));
	}
	elseif (strpos($uri, dirname($_SERVER['SCRIPT_NAME'])) === 0)
	{
		$uri = substr($uri, strlen(dirname($_SERVER['SCRIPT_NAME'])));
	}

	// This section ensures that even on servers that require the URI to be in the query string (Nginx) a correct
	// URI is found, and also fixes the QUERY_STRING server var and $_GET array.
	if (strncmp($uri, '?/', 2) === 0)
	{
		$uri = substr($uri, 2);
	}
	$parts = preg_split('#\?#i', $uri, 2);
	$uri = $parts[0];
	if (isset($parts[1]))
	{
		$_SERVER['QUERY_STRING'] = $parts[1];
		parse_str($_SERVER['QUERY_STRING'], $_GET);
	}
	else
	{
		$_SERVER['QUERY_STRING'] = '';
		$_GET = array();
	}

	if ($uri == '/' || empty($uri))
	{
		return '/';
	}

	$uri = parse_url($uri, PHP_URL_PATH);

	// Do some final cleaning of the URI and return it
	return str_replace(array('//', '../'), '/', trim($uri, '/'));
}

If the page requested was:

http://test.com/sample.php/easy/task

What would this evaluate to?

$_SERVER['REQUEST_URI']

It evaluates to:

/sample.php/easy/task

What would this evaluate to?

$_SERVER['SCRIPT_NAME']

It evaluates to:

/sample.php
Posted in CI Source Code Explained | Leave a comment

Utf8.php CI Explained

I’ve stopped work on this series to start a new one for CI 3.

Utf8.php is a core CodeIgniter class. This post is part of a series which explains the CodeIgniter (CI) source code. This post explains the CI 2.1.3 Utf8.php.

./system/core/Utf8.php

The Script’s Boilerplate

<?php  if ( ! defined('BASEPATH')) exit('No direct script access allowed');
/**
 * CodeIgniter
 *
 * An open source application development framework for PHP 5.1.6 or newer
 *
 * @package		CodeIgniter
 * @author		ExpressionEngine Dev Team
 * @copyright	Copyright (c) 2008 - 2011, EllisLab, Inc.
 * @license		http://codeigniter.com/user_guide/license.html
 * @link		http://codeigniter.com
 * @since		Version 2.0
 * @filesource
 */

// ------------------------------------------------------------------------

/**
 * Utf8 Class
 *
 * Provides support for UTF-8 environments
 *
 * @package		CodeIgniter
 * @subpackage	Libraries
 * @category	UTF-8
 * @author		ExpressionEngine Dev Team
 * @link		http://codeigniter.com/user_guide/libraries/utf8.html
 */
class CI_Utf8 {

Code: __construct()

/**
 * Constructor
 *
 * Determines if UTF-8 support is to be enabled
 *
 */
function __construct()
{
	log_message('debug', "Utf8 Class Initialized");

	global $CFG;

	if (
		preg_match('/./u', 'é') === 1					// PCRE must support UTF-8
		AND function_exists('iconv')					// iconv must be installed
		AND ini_get('mbstring.func_overload') != 1		// Multibyte string function overloading cannot be enabled
		AND $CFG->item('charset') == 'UTF-8'			// Application charset must be UTF-8
		)
	{
		log_message('debug', "UTF-8 Support Enabled");

		define('UTF8_ENABLED', TRUE);

		// set internal encoding for multibyte string functions if necessary
		// and set a flag so we don't have to repeatedly use extension_loaded()
		// or function_exists()
		if (extension_loaded('mbstring'))
		{
			define('MB_ENABLED', TRUE);
			mb_internal_encoding('UTF-8');
		}
		else
		{
			define('MB_ENABLED', FALSE);
		}
	}
	else
	{
		log_message('debug', "UTF-8 Support Disabled");
		define('UTF8_ENABLED', FALSE);
	}
}

Looking at the constructor piece-by-piece:

/**
 * Constructor
 *
 * Determines if UTF-8 support is to be enabled
 *
 */
function __construct()
{
	log_message('debug', "Utf8 Class Initialized");

	global $CFG;

The code above is self explanatory.

if (
	preg_match('/./u', 'é') === 1					// PCRE must support UTF-8
	AND function_exists('iconv')					// iconv must be installed
	AND ini_get('mbstring.func_overload') != 1		// Multibyte string function overloading cannot be enabled
	AND $CFG->item('charset') == 'UTF-8'			// Application charset must be UTF-8
	)
{

Q: What does this do?

preg_match('/./u', 'é') === 1

The dot meta-character matches to a single character.

The u pattern modifier says the pattern is to be treated as UTF-8.

preg_match() returns 1 if the pattern matches given subject, 0 if it does not, or FALSE if an error occurred.

These conditions are TRUE if the environment is UTF-8 friendly.

So, assuming it is UTF-8 friendly:

log_message('debug', "UTF-8 Support Enabled");
define('UTF8_ENABLED', TRUE);

Record the UTF-8 friendly status in log and as a constant.

// set internal encoding for multibyte string functions if necessary
// and set a flag so we don't have to repeatedly use extension_loaded()
// or function_exists()
if (extension_loaded('mbstring'))
{
	define('MB_ENABLED', TRUE);
	mb_internal_encoding('UTF-8');
}
else
{
	define('MB_ENABLED', FALSE);
}

Establish whether or not the mbstring extension is loaded; and, set internal encoding for multibyte string functions if necessary.

Otherwise:

else
{
	log_message('debug', "UTF-8 Support Disabled");
	define('UTF8_ENABLED', FALSE);
}	

Log the fact that "UTF-8 Support Disabled" and set the UTF8_ENABLED flag to FALSE.

Code: clean_string()

/**
 * Clean UTF-8 strings
 *
 * Ensures strings are UTF-8
 *
 * @access	public
 * @param	string
 * @return	string
 */
function clean_string($str)
{
	if ($this->_is_ascii($str) === FALSE)
	{
		$str = @iconv('UTF-8', 'UTF-8//IGNORE', $str);
	}

	return $str;
}

Q: What does this do?

iconv('UTF-8', 'UTF-8//IGNORE', $str)

This will remove any non-UTF-8 characters from $str. See manual page.

Code: safe_ascii_for_xml()

/**
 * Remove ASCII control characters
 *
 * Removes all ASCII control characters except horizontal tabs,
 * line feeds, and carriage returns, as all others can cause
 * problems in XML
 *
 * @access	public
 * @param	string
 * @return	string
 */
function safe_ascii_for_xml($str)
{
	return remove_invisible_characters($str, FALSE);
}

Q: What does this do?

remove_invisible_characters($str, FALSE)

remove_invisible_characters() is a CI common function — it is NOT a PHP function.

The FALSE specifies that $str is not url-encoded.

As the comment in the code says: Removes all ASCII control characters except horizontal tabs, line feeds, and carriage returns, as all others can cause problems in XML.

Code: convert_to_utf8()

/**
 * Convert to UTF-8
 *
 * Attempts to convert a string to UTF-8
 *
 * @access	public
 * @param	string
 * @param	string	- input encoding
 * @return	string
 */
function convert_to_utf8($str, $encoding)
{
	if (function_exists('iconv'))
	{
		$str = @iconv($encoding, 'UTF-8', $str);
	}
	elseif (function_exists('mb_convert_encoding'))
	{
		$str = @mb_convert_encoding($str, 'UTF-8', $encoding);
	}
	else
	{
		return FALSE;
	}

	return $str;
}

Code:

/**
 * Is ASCII?
 *
 * Tests if a string is standard 7-bit ASCII or not
 *
 * @access	public
 * @param	string
 * @return	bool
 */
function _is_ascii($str)
{
	return (preg_match('/[^\x00-\x7F]/S', $str) == 0);
}

preg_match() returns 1 if the pattern matches given subject, 0 if it does not, or FALSE if an error occurred.

What does the pattern /[^\x00-\x7F]/S match?

The S modifier has this description:

When a pattern is going to be used several times, it is worth spending more time analyzing it in order to speed up the time taken for matching. If this modifier is set, then this extra analysis is performed. At present, studying a pattern is useful only for non-anchored patterns that do not have a single fixed starting character.

What does the carrot/circumflex (^) inside the character class signify?

A character class matches a single character in the subject; the character must be in the set of characters defined by the class, unless the first character in the class is a circumflex, in which case the subject character must not be in the set defined by the class. If a circumflex is actually required as a member of the class, ensure it is not the first character, or escape it with a backslash.

x00-\x7F is the range for ASCII characters.

Hence, _is_ascii($str) returns TRUE if no non ASCII bytes exist in $str. FALSE otherwise.

Posted in CI Source Code Explained | Leave a comment