Thursday, 23 October 2014

C++ Correct multi threaded singleton initialisation

Correct double-checked locking pattern

1. Pointer must be atomic
2. Check, lock, check, construct

std::atomic<Foo*> foo { nullptr };

Foo* instance()
{
    Foo* f = foo; // single load of foo
    if (!f)
    {
        std::lock_guard<std::mutex> l(foo_lock);
        if (!foo)
        {
            foo = f = new Foo(); // assign both foo and f
        }
    }
    return f;
}

Even better, use std::unique_ptr and std::once to get automatic cleanup and less scaffolding

class Foo
{
public:
    static Foo& instance()
    {
        std::call_once(_create, [=]{
            _instance = std::make_unique<Foo>();
        });
        return *_instance;
    }

private:
    static std::unique_ptr<Foo> _instance;
    static std::once_flag       _create;
};

Or just use a function local static

Foo& Foo::instance()
{
    static Foo foo;
    return foo;
}

C++ decltype and auto type deduction

auto type deduction strips const, volatile and ref

const int& bar = foo;
auto baz = bar; // strips const and ref - therefore type of baz is int

decltype type deduction doesn't strip const, volatile and ref

// decltype of a name

const int& bar = foo;
decltype(bar) // does not strip const and ref - therefore type is const int&

// decltype of an expression
decltype(lvalue expression) always returns an lvalue reference

int arr[5];
arr[0] = 5;
decltype(arr[0]) // lvalue reference, therefore type is int&



C++ type information at run time

std::type_info::name and typeid(T).name() will give incorrect results, as required by the standard

use Boost.TypeIndex

#include <boost/type_index.hpp>

boost::type_index::type_id_with_cvr<T>().pretty_name();
boost::type_index::type_id_with_cvr<decltype(t)>().pretty_name();

C++14 mutable lambda and by-value and by-value init capture

by-value capture vs by-value init capture

by-value capture: type of `i` is `const int`
{
const int i = 0;
auto lambda = [i]() { };
}

by-value init capture: type of `i` is `int`
{
const int i = 0;
auto lambda = [i=i]() { }; 
}

lambda function call operator is const

error: by-value capture: type of `i` is `int`, but default lambda operator() is const member function
{
int i = 0;
auto lambda = [i]() { i = 1; }; 
}

error: by-value init capture: type of `i` is `int`, but default lambda operator() is const member function
{
const int i = 0;
auto lambda = [i=i]() { i = 1; }; 
}

making lambda function call operator mutable

error: by-value capture: type of `i` is `const int`, can't assign, even though lambda operator() is mutable member function
{
const int i = 0;
auto lambda = [i]() mutable { i = 1; };  
}

by-value capture: type of `i` is `int`, and lambda operator() is mutable member function
{
int i = 0;
auto lambda = [i]() mutable { i = 1; };  
}

by-value init capture: type of `i` is `int`, and lambda operator() is mutable member function
{
const int i = 0;
auto lambda = [i=i]() mutable { i = 1; }; 
}

Monday, 18 August 2014

Python for data anlysis

numpy

Array creation functions

array: Convert input data (list, tuple, array, or other sequence type) to an ndarray either by inferring a dtype or explicitly specifying a dtype. Copies the input data by default.
asarray: Convert input to ndarray, but do not copy if the input is already an ndarray
arange: Like the built-in range but returns an ndarray instead of a list.
ones, ones_like: Produce an array of all 1’s with the given shape and dtype. ones_like takes another array and produces a ones array of the same shape and dtype.
zeros, zeros_like: Like ones and ones_like but producing arrays of 0’s instead
empty, empty_like: Create new arrays by allocating new memory, but do not populate with any values like ones and zeros
eye, identity: Create a square N x N identity matrix (1’s on the diagonal and 0’s elsewhere)

Unary ufuncs

abs, fabs: Compute the absolute value element-wise for integer, floating point, or complex values. Use fabs as a faster alternative for non-complex-valued data
sqrt: Compute the square root of each element. Equivalent to arr ** 0.5
square: Compute the square of each element. Equivalent to arr ** 2
exp: Compute the exponent e^x of each element
log, log10, log2, log1p: Natural logarithm (base e), log base 10, log base 2, and log(1 + x), respectively
sign: Compute the sign of each element: 1 (positive), 0 (zero), or -1 (negative)
ceil: Compute the ceiling of each element, i.e. the smallest integer greater than or equal to
each element
floor: Compute the floor of each element, i.e. the largest integer less than or equal to each
element
rint: Round elements to the nearest integer, preserving the dtype
modf: Return fractional and integral parts of array as separate array
isnan: Return boolean array indicating whether each value is NaN (Not a Number)
isfinite, isinf: Return boolean array indicating whether each element is finite (non-inf, non-NaN) or infinite, respectively
cos, cosh, sin, sinh, tan, tanh: Regular and hyperbolic trigonometric functions
arccos, arccosh, arcsin, arcsinh, arctan, arctanh: Inverse trigonometric functions
logical_not: Compute truth value of not x element-wise. Equivalent to -arr.

Binary universal functions

add: Add corresponding elements in arrays
subtract: Subtract elements in second array from first array
multiply: Multiply array elements
divide, floor_divide: Divide or floor divide (truncating the remainder)
power: Raise elements in first array to powers indicated in second array
maximum, fmax: Element-wise maximum. fmax ignores NaN
minimum, fmin: Element-wise minimum. fmin ignores NaN
mod: Element-wise modulus (remainder of division)
copysign: Copy sign of values in second argument to values in first argument
greater, greater_equal, less, less_equal, equal, not_equal: Perform element-wise comparison, yielding boolean array. Equivalent to infix operators >, >=, <, <=, ==, !=
logical_and, logical_or, logical_xor: Compute element-wise truth value of logical operation. Equivalent to infix operators & |, ^

Basic array statistical methods

sum: Sum of all the elements in the array or along an axis. Zero-length arrays have sum 0.
mean: Arithmetic mean. Zero-length arrays have NaN mean.
std, var: Standard deviation and variance, respectively, with optional degrees of freedom adjustment (default denominator n).
min, max: Minimum and maximum.
argmin, argmax: Indices of minimum and maximum elements, respectively.
cumsum: Cumulative sum of elements starting from 0
cumprod: Cumulative product of elements starting from 1

Array set operations

unique(x): Compute the sorted, unique elements in x
intersect1d(x, y): Compute the sorted, common elements in x and y
union1d(x, y): Compute the sorted union of elements
in1d(x, y): Compute a boolean array indicating whether each element of x is contained in y
setdiff1d(x, y): Set difference, elements in x that are not in y
setxor1d(x, y): Set symmetric differences; elements that are in either of the arrays, but not both

Linear Algebra

diag: Return the diagonal (or off-diagonal) elements of a square matrix as a 1D array, or convert a 1D array into a square matrix with zeros on the off-diagonal
dot: Matrix multiplication
trace: Compute the sum of the diagonal elements
det: Compute the matrix determinant
eig: Compute the eigenvalues and eigenvectors of a square matrix
inv: Compute the inverse of a square matrix
pinv: Compute the Moore-Penrose pseudo-inverse inverse of a square matrix
qr: Compute the QR decomposition
svd: Compute the singular value decomposition (SVD)
solve: Solve the linear system Ax = b for x, where A is a square matrix
lstsq: Compute the least-squares solution to y = Xb

Random Number Generation

seed: Seed the random number generator
permutation: Return a random permutation of a sequence, or return a permuted range
shuffle: Randomly permute a sequence in place
rand: Draw samples from a uniform distribution
randint: Draw random integers from a given low-to-high range
randn: Draw samples from a normal distribution with mean 0 and standard deviation 1 (MATLAB-like interface)
binomial: Draw samples a binomial distribution
normal: Draw samples from a normal (Gaussian) distribution
beta: Draw samples from a beta distribution
chisquare: Draw samples from a chi-square distribution
gamma: Draw samples from a gamma distribution
uniform: Draw samples from a uniform [0, 1) distribution
Taken from Python for Data Anlysis by Wes McKinney

Sunday, 17 August 2014

IPython debugging

Debugger commands

  • h (help) Display command list
  • help command Show documentation for command
  • c (continue) Resume program execution
  • q (quit) Exit debugger without executing any more code
  • b (break) number Set breakpoint at number in current file
  • b path/to/file.py:number Set breakpoint at line number in specified file
  • s (step) Step into function call
  • n (next) Execute current line and advance to next line at current level
  • u/d (up) / (down) Move up/down in function call stack
  • a (args) Show arguments for current function
  • debug statement Invoke statement statement in new (recursive) debugger
  • l (list) statement Show current position and context at current level of stack
  • w (where) Print full stack trace with context at current position

Post-mortem debugging

%debug

Entering %debug immediately after an exception has occurred drops you into the stack frame where the exception was raised

Utility functions

Poor man’s breakpoint

def set_trace():
    from IPython.core.debugger import Pdb
    Pdb(color_scheme='Linux').set_trace(sys._getframe().f_back)

Putting set_trace() in your code will automatically drop into the debugger when the line is executed.

Interactive function debugging

def debug(f, *args, **kwargs):
    from IPython.core.debugger import Pdb
    pdb = Pdb(color_scheme='Linux')
    return pdb.runcall(f, *args, **kwargs)

Passing a function to debug will drop you into the debugger for an arbitrary function call.

debug(fn, arg1, arg2, arg3, kwarg=foo, kwarg=bar)

Interactive script debugging

Executing a script via %run with -d will start the script in the debugger

%run -d ./my_script.py

Specifying a line number with -b starts the script with a breakpoint already set

%run -d -b20 ./my_script.py # sets a breakpoint on line 20

Taken from Python for Data Anlysis by Wes McKinney

Sunday, 29 June 2014

bjam / boost.build

Boost.Build
Common signature:

rule rule-name
    (
        target-name :
        sources + :
        requirements * :
        default-build * :
        usage-requirements *
    )

target-name is the name used to request the target
sources is the list of source files or other targets
requirements is the list of properties that must always be present when building this target
default-build is the list of properties that will be used unless some other value is already specified (eg: on cmd line or propagation from a dependent target)
usage-requirements is the properties that will be propagated to all targets that use this one

Helper commands:
glob - takes a list shell pattern and returns the list of files in the project's source directory that match the pattern. optional second argument is a list of exclude patterns
lib tools : [ glob *.cpp : exclude.cpp ] ;

glob-tree - recursive glob
lib tools : [ glob-tree *.cpp : .svn ] ;

constant - project wide constant
constant VERSION : 1.34.0 ;

Project:

project project-name
    : requirements <feature>value <feature>value
    ;

Programs:

exe app-name
    : app.cpp some_library.lib ../project//library
    : <threading>multi
    ;

sources is one cpp file (app.cpp), a library in the same directory (some_library.lib) and a Jamfile target (library) specified in the Jamfile found in the path ../project
requirements is that threading is multi

Libraries:
Library targets can represent:

Libraries that should be built from source


lib lib-name
    : lib.cpp
    ;

sources is one cpp file (lib.cpp)

Prebuilt libraries which already exist on the system
Such libraries can be searched for by the tools using them (typically with the linker's -l option), or their paths can be known in advance by the build system.

lib z
    :
    : <name>z <search>../3rd/libz
    ;

lib compress
    :
    : <file>/opt/libs/libcompress.a
    ;

<name> specifies the name of the library without the standard prefixes and suffixes.
In the above example, z could refer to z.so, libz.a, z.lib etc
<search> specifies paths in which to search for the library (in addition to the default compiler paths)
<search> can be specified multiple times, or omitted (meaning only the default compiler paths will be searched)
Note that <search> paths are added to the linker search path (-L) for all libraries being linked in the target, which can potentially lead to libraries from another path being picked up first

Convenience helper syntax for prebuilt libraries

lib z ;
lib gui db aux ;

is the same as

lib z : : <name>z ;
lib gui : : <name>gui ;
lib db : : <name>db ;
lib aux : : <name>aux ;

Prebuilt libraries for different build variants

lib foo
    :
    : <file>libfoo_release.a <variant>release
    ;

lib foo
    :
    : <file>libfoo_debug.a <variant>debug

    ;

Referencing other libraries
When a library references another library, that library should be listed in its list of sources.
Specify library dependencies even for searched and prebuilt libraries

lib z ;
lib png : z : <name>png ;

How Boost.Build includes library dependencies
When a library has a shared library as a source, or a static library has another static library as a source, then an target linking to the first library will also automatically link to the source library

However, when a shared library has a static library as a source, then the shared library will be built such that it completely includes the static library (--whole-archive)
If you don't want this behaviour, you need to use the following:

lib a : a.cpp : <use>b : : <library>b ;

This says that library uses library b, and causes executables that link to a to also link to b, instead of a referring to b

Automatically add a library's header location to any upstream target's include path
When a library's interface is in a header file, you can set usage-requirements for the library to include the path where the header file is, so that any target using the library target will automatically get the path to its header added to its include search path

lib foo : foo.cpp : : : <include>. ;

Control library linking order
If library a "uses" library b, then library a will appear before library b.
Library a is considered to use library b is b is present either in library a's sources or its usage is listed in its requirements
The <use> feature can also be used to explicitly express a relationship.

lib z ;
lib png : : <use>z ;
exe viewer : viewer png z ;

z will be linked before png

Special helper for zlib.
zlib can be configured either to use precompiled binaries or to build the library from source.

Find zlib in the default system location
using zlib ;
Build zlib from source
using zlib : 1.2.7 : <source>/home/steven/zlib-1.2.7 ;
Find zlib in /usr/local
using zlib : 1.2.7 : <include>/usr/local/include <search>/usr/local/lib ;
Build zlib from source for msvc and find prebuilt binaries for gcc.
using zlib : 1.2.7 : <source>C:/Devel/src/zlib-1.2.7 : <toolset>msvc ;
using zlib : 1.2.7 : : <toolset>gcc ;

Builtin features:
variant - build variant. Default configuration values: debug, release, profile.
link - library linking. values: shared, static
runtime-link - binary linking. values: shared, static
threading - link additional threading libraries. values: single, multi
source - useful for adding the same source to all targets in the project (put <source> in requirements), or to conditionally include a source
library - useful for linking to the same libraries for all targets in the project
dependency - introduces a dependency on the target named by the value. If the declared target is built, the dependent target will be too
implicit-dependency - indicates the target named by the value may produce files which the declared target uses.
use - introduces a dependency on the target named by the value, and adds its usage requirements to the build properties of the target being declared. The dependency is not used in any other way.
dll-path - add a shared library search path.
hardcode-dll-path - hardcode the dll-path entries. Values: true, false.
cflags, cxxflags, linkflags - passed on to the corresponding tools.
include - add an include search path.
define - define a preprocessor symbol. A value can be specified: <define>symbol=value
warnings - control the warning level of the compiler. Values: off, on, all.
warnings-as-errors - turn on to have builds fail when a warning is emitted.
build - skips building the target. Useful to conditionally set the value. Values: no.
tag - customize the name of generated files. Value: @rulename, where rulename is the name of a rule with the signature: rule tag ( name : type ? : property-set ). The rule will be called for each target with the default name of the target, the type of the target, and property set. Return an empty string to use the default target name, or a non empty string to be used for the name of the target. Useful for encoding library version nos etc.
debug-symbols - Include debug symbols in the object files etc. Values: on, off.

Objects:
Change behaviour for only a single object file

obj foo : foo.cpp : <optimizarion>off ;
exe bar : bar.cpp foo ;

foo will be built with the special flags, and then van be pulled into other targets

Alias:
Alternative name for a group of targets

alias core : foo bar baz ;

Using core in the source list of any other target or on the command line will translate to the aliased group of targets

Change build properties

alias my_bar : ../foo//bar : <link>static ;

my_bar now refers to the bar target in the foo Jamfile, but has the requirement that it be linked statically

Specify a header only library

alias hdr_only_lib : : : : <include>/path/to/headers ;

Using hdr_only_lib will just add an include path to any targets

Propagation of usage-requirements
When an alias has sources, the usage-requirements of those sources are propagated as well.

lib lib1 : lib1src.cpp : : : <include>/path/to/lib1.hpp ;
lib lib2 : lib2src.cpp : : : <include>/path/to/lib2.hpp ;
alias static_libs : lib1 lib2 : <link>static ;
exe main : main.cpp static_libs ;

Compile main with lib1 and lib2 as static libraries, and their paths are added to the include search path

Installing:
Installing a built target to a relative path

install dist : foo bar ;

foo and bar will be moved to the dist folder, relative to the Jamfile's directory

Installing a built target to specific location

install dist : foo bar : <location>/install/path/location

foo and bar will be moved to /install/path/location

Installing a built target to a path based on a conditional expression
(see conditional expressions below)

install dist 
    : foo bar 
    : <variant>release:<location>dist/release
      <variant>debug:<location>dist/debug ;

foo and bar will be installed to relative path dist/<build-variant>

Installing a built target to a path based on an environment variable
(see accessing environment variables below)

install dist : foo bar : <location>$(DIST) ;

Automatically install all dependencies

install dist 
    : foo
    : <install-dependencies>on
      <install-type>EXE
      <install-type>LIB
    ;

will find all targets foo depends on, and install those which are either executables or libraries.

Preserve directory hierarchy

install headers 
    : a/b/c.h
    : <location>/tmp
      <install-source-root>a
    ;

/tmp/b/c.h will be installed

Install into several directories
use an alias rule to install to several directories

alias install : install-bin install-lib ;
install install-bin : apps : <location>/usr/bin ;
install install-lib : libs : <location>/usr/lib ;

set the RPATH

install installed : application : <dll-path>/usr/lib/snake
                                  <location>/usr/bin ;

will allow the application to find libraries placed in the /usr/lib/snake directory.

Testing:
unit-testing

unit-test foo_test : test.cpp foo ;

behaves just like exe rule, but the test is automatically run after building

testing through another application

unit-test foo_test : test.cpp foo : <testing.launcher>valgrind ;

runs the test through the launcher, eg: valgrind build/path/foo_test

Environment variables:
local foo = [ SHELL "bar" ] ;

Executing external programs:
import os ;
local SOME_PATH = [ os.environ SOME_PATH ] ;
exe foo : foo.cpp : <include>$(SOME_PATH) ;

Conditional expressions:
syntax

property ( "," property ) * ":" property

multiple properties can be combined

exe hello : hello.cpp : <os>NT,<toolset>gcc:<link>static ;

will link hello statically only when compiling with gcc on NT

Command reference:
http://www.boost.org/doc/libs/1_55_0/doc/html/bbv2/reference.html