ssh tunnel

Create a tunnel to a remote host via a gateway

    ssh -L <local-port-to-listen>:<destination-host>:<destination-port> <gateway_user>@<gateway>

This opens a local port listening for traffic on <local-port-to-listen> and forwards that traffic via the gateway (user@gateway) to the remote destination <destination-host>:<destination-port>

-f executes ssh in the background
-N means no remote command (ie: just create a tunnel)

    ssh -N -f -L 8080:destination:8080 user@gateway

Pointing your browser to localhost:8080 will connect to the ssh tunnel which forwards the data to destination:8080, going via the gateway

Multicast troubleshooting

Troubleshooting multicast:

Check that the interface is configured with multicast:

$ ifconfig eth9.240
eth9.240 Link encap:Ethernet HWaddr 00:60:DD:44:67:9E
inet addr:10.185.131.41 Bcast:10.185.131.63 Mask:255.255.255.224
inet6 addr: fe80::260:ddff:fe44:679e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1

Check that the multicast addresses you are subscribing to have a route to that particular interface:

$ ip route
224.0.0.0/4 dev eth9.240 scope link

Run your application and check if the subscriptions are going to the correct interface:

$ netstat -g
IPv6/IPv4 Group Memberships
Interface RefCnt Group
[...]
eth9.240 1 239.1.127.215
eth9.240 1 239.1.1.215

Run tcpdump and check that you are indeed receiving traffic. Do this while your application is running; otherwise the igmp subscription will not be on.

$ tcpdump -i eth9.240
10:15:13.385228 IP 10.0.8.121.45666 > 239.1.1.1.51001: UDP, length 16

If you got to the tcpdump part, the networking should be OK.

If your application is still not receiving packets, it is probably because of the rp_filter in Linux. 

The rp_filter filters out any packets that do not have a route on a particular interface. In the example above, if 10.0.8.121 is not routable via eth9.240 so the solution is to:

Check the filter
$ cat /proc/sys/net/ipv4/conf/ethX/rp_filter

add this line to /etc/sysctl.conf
    net.ipv4.conf.eth9/240.rp_filter = 0
$ sudo sysctl -p

Check if it’s OK
$ sysctl -a | grep “eth9/240.rp_filter”

C++ Correct multi threaded singleton initialisation

Correct double-checked locking pattern

1. Pointer must be atomic
2. Check, lock, check, construct

std::atomic<Foo*> foo { nullptr };

Foo* instance()
{
    Foo* f = foo; // single load of foo
    if (!f)
    {
        std::lock_guard<std::mutex> l(foo_lock);
        if (!foo)
        {
            foo = f = new Foo(); // assign both foo and f
        }
    }
    return f;
}

Even better, use std::unique_ptr and std::once to get automatic cleanup and less scaffolding

class Foo

{

public:

    static Foo& instance()

    {

        std::call_once(_create, [=]{

            _instance = std::make_unique<Foo>();

        });

        return *_instance;

    }

private:

    static std::unique_ptr<Foo> _instance;

    static std::once_flag       _create;

};

Or just use a function local static

Foo& Foo::instance()

{

    static Foo foo;

    return foo;

}

C++ decltype and auto type deduction

auto type deduction strips const, volatile and ref

const int& bar = foo;
auto baz = bar; // strips const and ref - therefore type of baz is int

decltype type deduction doesn't strip const, volatile and ref

// decltype of a name

const int& bar = foo;
decltype(bar) // does not strip const and ref - therefore type is const int&

// decltype of an expression

decltype(lvalue expression) always returns an lvalue reference

int arr[5];

arr[0] = 5;

decltype(arr[0]) // lvalue reference, therefore type is int&

C++ type information at run time

std::type_info::name and typeid(T).name() will give incorrect results, as required by the standard

use Boost.TypeIndex

#include <boost/type_index.hpp>

boost::type_index::type_id_with_cvr<T>().pretty_name();
boost::type_index::type_id_with_cvr<decltype(t)>().pretty_name();

C++14 mutable lambda and by-value and by-value init capture

by-value capture vs by-value init capture

by-value capture: type of `i` is `const int`
{
const int i = 0;
auto lambda = [i]() { };
}

by-value init capture: type of `i` is `int`
{
const int i = 0;
auto lambda = [i=i]() { }; 
}

lambda function call operator is const

error: by-value capture: type of `i` is `int`, but default lambda operator() is const member function
{
int i = 0;
auto lambda = [i]() { i = 1; }; 
}

error: by-value init capture: type of `i` is `int`, but default lambda operator() is const member function
{
const int i = 0;
auto lambda = [i=i]() { i = 1; }; 
}

making lambda function call operator mutable

error: by-value capture: type of `i` is `const int`, can't assign, even though lambda operator() is mutable member function
{
const int i = 0;
auto lambda = [i]() mutable { i = 1; };  
}

by-value capture: type of `i` is `int`, and lambda operator() is mutable member function
{
int i = 0;
auto lambda = [i]() mutable { i = 1; };  
}

by-value init capture: type of `i` is `int`, and lambda operator() is mutable member function
{
const int i = 0;
auto lambda = [i=i]() mutable { i = 1; }; 
}

Python for data anlysis

numpy

Array creation functions

array: Convert input data (list, tuple, array, or other sequence type) to an ndarray either by inferring a dtype or explicitly specifying a dtype. Copies the input data by default.
asarray: Convert input to ndarray, but do not copy if the input is already an ndarray
arange: Like the built-in range but returns an ndarray instead of a list.
ones, ones_like: Produce an array of all 1’s with the given shape and dtype. ones_like takes another array and produces a ones array of the same shape and dtype.
zeros, zeros_like: Like ones and ones_like but producing arrays of 0’s instead
empty, empty_like: Create new arrays by allocating new memory, but do not populate with any values like ones and zeros
eye, identity: Create a square N x N identity matrix (1’s on the diagonal and 0’s elsewhere)

Unary ufuncs

abs, fabs: Compute the absolute value element-wise for integer, floating point, or complex values. Use fabs as a faster alternative for non-complex-valued data
sqrt: Compute the square root of each element. Equivalent to arr ** 0.5
square: Compute the square of each element. Equivalent to arr ** 2
exp: Compute the exponent e^x of each element
log, log10, log2, log1p: Natural logarithm (base e), log base 10, log base 2, and log(1 + x), respectively
sign: Compute the sign of each element: 1 (positive), 0 (zero), or -1 (negative)
ceil: Compute the ceiling of each element, i.e. the smallest integer greater than or equal to
each element
floor: Compute the floor of each element, i.e. the largest integer less than or equal to each
element
rint: Round elements to the nearest integer, preserving the dtype
modf: Return fractional and integral parts of array as separate array
isnan: Return boolean array indicating whether each value is NaN (Not a Number)
isfinite, isinf: Return boolean array indicating whether each element is finite (non-inf, non-NaN) or infinite, respectively
cos, cosh, sin, sinh, tan, tanh: Regular and hyperbolic trigonometric functions
arccos, arccosh, arcsin, arcsinh, arctan, arctanh: Inverse trigonometric functions
logical_not: Compute truth value of not x element-wise. Equivalent to -arr.

Binary universal functions

add: Add corresponding elements in arrays
subtract: Subtract elements in second array from first array
multiply: Multiply array elements
divide, floor_divide: Divide or floor divide (truncating the remainder)
power: Raise elements in first array to powers indicated in second array
maximum, fmax: Element-wise maximum. fmax ignores NaN
minimum, fmin: Element-wise minimum. fmin ignores NaN
mod: Element-wise modulus (remainder of division)
copysign: Copy sign of values in second argument to values in first argument
greater, greater_equal, less, less_equal, equal, not_equal: Perform element-wise comparison, yielding boolean array. Equivalent to infix operators >, >=, <, <=, ==, !=
logical_and, logical_or, logical_xor: Compute element-wise truth value of logical operation. Equivalent to infix operators & |, ^

Basic array statistical methods

sum: Sum of all the elements in the array or along an axis. Zero-length arrays have sum 0.
mean: Arithmetic mean. Zero-length arrays have NaN mean.
std, var: Standard deviation and variance, respectively, with optional degrees of freedom adjustment (default denominator n).
min, max: Minimum and maximum.
argmin, argmax: Indices of minimum and maximum elements, respectively.
cumsum: Cumulative sum of elements starting from 0
cumprod: Cumulative product of elements starting from 1

Array set operations

unique(x): Compute the sorted, unique elements in x
intersect1d(x, y): Compute the sorted, common elements in x and y
union1d(x, y): Compute the sorted union of elements
in1d(x, y): Compute a boolean array indicating whether each element of x is contained in y
setdiff1d(x, y): Set difference, elements in x that are not in y
setxor1d(x, y): Set symmetric differences; elements that are in either of the arrays, but not both

Linear Algebra

diag: Return the diagonal (or off-diagonal) elements of a square matrix as a 1D array, or convert a 1D array into a square matrix with zeros on the off-diagonal
dot: Matrix multiplication
trace: Compute the sum of the diagonal elements
det: Compute the matrix determinant
eig: Compute the eigenvalues and eigenvectors of a square matrix
inv: Compute the inverse of a square matrix
pinv: Compute the Moore-Penrose pseudo-inverse inverse of a square matrix
qr: Compute the QR decomposition
svd: Compute the singular value decomposition (SVD)
solve: Solve the linear system Ax = b for x, where A is a square matrix
lstsq: Compute the least-squares solution to y = Xb

Random Number Generation

seed: Seed the random number generator
permutation: Return a random permutation of a sequence, or return a permuted range
shuffle: Randomly permute a sequence in place
rand: Draw samples from a uniform distribution
randint: Draw random integers from a given low-to-high range
randn: Draw samples from a normal distribution with mean 0 and standard deviation 1 (MATLAB-like interface)
binomial: Draw samples a binomial distribution
normal: Draw samples from a normal (Gaussian) distribution
beta: Draw samples from a beta distribution
chisquare: Draw samples from a chi-square distribution
gamma: Draw samples from a gamma distribution
uniform: Draw samples from a uniform [0, 1) distribution
Taken from Python for Data Anlysis by Wes McKinney