2018/10/19

Shell script for continuous integration for iOS projects

I've done this various times in so different ways and for different services but in the end internally everything boils down to the xcodebuild command to build and test. Also generate unit tests and coverage report. Here is a not-fancy but effective and short way of doing it (Yeah, I know recently Circle, Travis, etc, even Azure Devops are more exciting these days).

Build and Test an Xcode project

export LC_CTYPE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
cd change/to/xcodeproj/directory

carthage version
carthage bootstrap --cache-builds --platform iOS
# pod install

MY_SCHEME_NAME='MySchemeName'
MY_PROJECT_FILENAME='MyProject.xcodeproj'

xcodebuild -version
xcodebuild clean test -scheme "$MY_SCHEME_NAME" \
    -sdk iphonesimulator \
    -project "$MY_PROJECT_FILENAME" \
    -configuration Debug \
    -destination "platform=iOS Simulator,name=iPhone 8" \
    -enableCodeCoverage YES | tee xcodebuild_${BUILD_NUMBER}.log |xcpretty --report junit

slather version
slather coverage --verbose --jenkins --arch x86_64 \
    --scheme "$MY_SCHEME_NAME" --input-format profdata \
    --cobertura-xml --output-directory build/reports \
    --show "$MY_PROJECT_FILENAME"

Explanation & Recommendations

  • Make sure to set LC_ALL to UTF8 because sometimes xcpretty cannot parse some special characters written by xcodebuild (coming from your code)
  • Make sure to print tools versions. Version numbers are helpful when problems occur and you want to reproduce CI environment locally or simply look for release notes or upgrade things.
  • xcpretty is nice but sometimes it cuts the error logs incorrectly, specially for UI Tests. That is why I used tee to write the raw log from xcodebuild to a file. Otherwise considering disabling xcpretty.
  • Disabling xcpretty means no junit report so Jenkins job might end in error.
  • When using slather make sure to set the architecture. It is required when Xcode builds more than only one architecture.
  • slather is cool but documentation is poor and some parameter don't work together. For example --simple-output and --cobertura-xml causes no errors and it silently skips cobertura xml output :(

Upload IPA file to iTunes connect

Usually after building and testing the app you want to upload the IPA file to iTunes connect for TestFlight testing.

/Applications/Application\ Loader.app/Contents/Frameworks/ITunesSoftwareService.framework/Support/altool --upload-app -u $USERNAME -p $PASSWORD -f $IPA_FILENAME
Hope it helps :]

2018/10/13

Installing Jenkins in Cloud Foundry PaaS

  1. Get Jenkins
    Go to https://jenkins.io/download/ and download the Generic Java package (.war).
  2. Deploy Jenkins
    We are going to deploy jenkins.war to a new instance. jenkins.war is a java app but cannot be deployed with the standard java-buildpack because it must be build for tomcat with another script. Hence, we need to use a custom buildpack and luckly there is one in github.com/Altoros/jenkins-buildpack.
    cf push \
        -p jenkins.war \
        -b https://github.com/Altoros/jenkins-buildpack \
        -m 1G \
        nacho-ci
    
    That is all for deploy, when accessing the app for first time a token will be asked so we should check logs or access /home/vcap/.jenkins/secrets/initialAdminPassword via ssh
    bx cf logs nacho-ci --recent
    
    Something like this will be shown:
    ...
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR INFO: 
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR Jenkins initial setup is required. An admin user has been created and a password generated.
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR Please use the following password to proceed to installation:
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR 9bbc7b5799e14847b4cc8be8c823cc2b
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR This may also be found at: /home/vcap/.jenkins/secrets/initialAdminPassword
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    2018-10-13T15:53:28.17+0900 [APP/PROC/WEB/0] ERR *************************************************************
    ...
    
Now copy and paste the token and Jenkins will start :)

2018/10/02

Convert issue to pull request with Github API

Here is an example for doing a pull request from fork: nacho4d branch: feature/fix-xcode-configure-file to meson to branch master.
Instead of creating a new shiny pull request I am converting an existing issue 4296 into a pull request. Which is nice because in that issue we were discussing all things I have done in the pull request :)
curl \
  --user "nacho4d:password-of-nacho4d" \
  --request POST \
  --data '{ \
      "issue":4296, \
      "head":"nacho4d:feature/fix-xcode-configure-file", \
      "base":"master"}' \
  https://api.github.com/repos/mesonbuild/meson/pulls
To make it more real, we should use avoid passing the password in this way (because user:password will be recorded in terminal history and potencially seen by someone else, etc).

Instead of --user user:password \ just omit the password and curl will ask for the password everytime or even better use an authorization token --header Authorization: YOUR_TOKEN_HERE' \

Github's official documentation:
Hope this helps :)

2018/09/23

Debugging meson

Recently I started using meson for personal mini projects. I have used CMake in the past and my personal impression is that things get rapidly complicated and I believe they don't need to be like so. I prefer meson for its simplicity.

These days meson build seems promising except its Xcode support is not very good (apparently not many Xcode users use meson). So I decided to fix it myself (at least the parts I know). I write my notes here since I am not very familiar with python and its development environment.

Notes on how to debug/fix issues with meson

  1. Clone meson
    git clone git@github.com:mesonbuild/meson.git
    
  2. Open meson directory in Visual Code
    code meson
    
  3. Create settings.json and launch.json
    • settings.json
      Create settings.json to make sure the correct python version (3.5 or above) is set for the workspace (and also because I want Visual Code stop asking me frequently if it should install a linter. I set it to no because I already set the git commit hook recommended for this project. )

      This is my .vscode/settings.json file:
      {
          "python.linting.pylintEnabled": false,
          "python.pythonPath": "/usr/local/bin/python3"
      }
      
    • launch.json
      Create launch.json to be able to start the debugger from within the editor window and be able to put breakpoints and inspect variables more easily.
      Simple create file .vscode/launch.json with below contents.
      FYI: To create the default launch,json file change to the debug pane (Shift + Cmd + D) and add a configuration; The file will be created with various debug configurations. Most of them are for server/client development and are not relevant to meson project, feel free to delete them if not needed.

      This is my .vscode/launch.json file:
      {
          "version": "0.2.0",
          "configurations": [
              // Configuration that uses meson.py to run one of the project tests.
              // Move to a test dir and execute `meson.py build --backend=xcode`.
              {
                  "name": "run project test 1",
                  "type": "python",
                  "request": "launch",
                  "program": "${workspaceFolder}/meson.py",
                  "console": "integratedTerminal",
                  "args": [
                      "build",
                      "--backend=xcode",
                  ],
                  "cwd": "${workspaceFolder}/test cases/common/1 trivial"
              },
              // Configuration that uses run_unittests.py to run a particular test.
              // Move to root directory and execute:
              // `run_unittests.py -v TestCase.testMethod`. Since `--backend=xcode`
              // cannot be set via an argument use an environment variable.
              {
                  "name": "run_unittests.py",
                  "type": "python",
                  "request": "launch",
                  "program": "${workspaceFolder}/run_unittests.py",
                  "console": "integratedTerminal",
                  "args": [
                      "-v",
                      "AllPlatformTests.test_always_prefer_c_compiler_for_asm",
                  ],
                  "cwd": "${workspaceFolder}",
                  "env": {
                      "MESON_UNIT_TEST_BACKEND":"xcode"
                  }
              },
          ]
      }
      
  4. Now just set your brake points and debug/code :)

2018/03/08

Using GnuPG

1. Installing GnuPG

On various Linux flavors it is apparently available by default. On macos we can brew it.
brew install gpg

2. Generate Keys

I strongly recommend not to use the short version --gen-key because it uses by default RSA algorithm which does not allow key signing. I recommend to use --full-generate-key and use kind "DSA and Elgamal" to generate the pair of keys (Public key and Private key). You will be required to enter a password, this password will be required every-time a message needs to be decrypted because it will use your private key.
gpg --full-generate-key
gpg (GnuPG) 2.2.4; Copyright (C) 2017 Free Software Foundation, Inc.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Please select what kind of key you want:
   (1) RSA and RSA (default)
   (2) DSA and Elgamal
   (3) DSA (sign only)
   (4) RSA (sign only)
Your selection? 2
DSA keys may be between 1024 and 3072 bits long.
What keysize do you want? (2048) 
Requested keysize is 2048 bits   
Please specify how long the key should be valid.
         0 = key does not expire
        = key expires in n days
      w = key expires in n weeks
      m = key expires in n months
      y = key expires in n years
Key is valid for? (0) 0
Key does not expire at all
Is this correct? (y/N) y
                        
GnuPG needs to construct a user ID to identify your key.

Real name: YOURID
Email address: yourid@email.com
Comment:                       
You selected this USER-ID:
    "YOURID "

Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? O

3. Exporting and importing keys

Export public key

To export in ASCII armor format:
gpg --output YOURID.pub.gpg --armor --export YOURID
To export in binary format simply skip the --armor parameter.

Export private key

Similar to public key, to export a private key in ASCII armor format:
gpg --output YOURID.sec.gpg --armor --export-secret-keys YOURID
To export in binary format simply skip the --armor parameter.

Import public key

To simply import the key we can use the --import command.
gpg --import OTHERID.pub.gpg
HOWEVER, at this point using this new key to decrypt something will lead to WARNings because the key is not verified, not trusted, not checked, etc. Please make sure you have the right key (check the fingerprint) and trust it. For full conceptual explanation see the official docs here and here
gpg --list-keys
gpg --edit-key OTHERID

> fpr
> sign
> check
> trust
> save
There is a good guide here too from Digital Ocean guys.

Import private key

We use the same --import command. gpg will recognize if key is private or public :)
gpg --import blake.secret.gpg
# (type password)
To see the list of private keys in the keyring:
gpg --list-secret-keys

Asymmetric encryption and decryption

Asymmetric cryptography means the message is encrypted with the public key and decrypted with the private key that is paired with that public key. This means, the sender cannot decrypt the message. Only the receiver can decrypt it (Off-course, unless the message was sent to self)

Encrypt a file

To asymmetrically encrypt a file we need to have the public key of the receiver in the keyring.
gpg --output file.doc.gpg --local-user SENDERID --armor --encrypt --sign --recipient RECEIVERID file.doc
or a shorter version
gpg --output file.doc.gpg -u SENDERID -r RECEIVERID -ase file.doc
We can skip the --armor or -a option to have encrypted files in binary format.
Not recommended but we can skip the --sign or -s option to not sign it.
If --local-user or -u is skipped then the default key will be used.

Decrypt a file

To asymmetrically decrypt a file we must have the private key that corresponds to the public key that was used at encryption time.
To decrypt and verify the sign:
gpg --output file.doc --decrypt file.doc.gpg
This command will requires two things to succeed:
  1. A kind of key able to sign (RSA does not do it, that is why I suggested to use DSA and Elgamal kind).
  2. Option --sign or -s should be passed at encryption time.
Not recommended but if you need to skip sign verification:
gpg --output file.doc --skip-verify --decrypt file.doc.pgp

Symmetric Encryption

Encrypt a file (Symmetric encryption)

Symmetric encryption requires just a password. Anyone with this password should be able to decrypt to file.
gpg --output file.doc.gpg --symmetric file.doc
(enter password)

More

2018/02/15

Convert between arrays and lists with streams with Java

There are several ways of converting between arrays and lists but there are some gotchas when converting with non-streams methods. In reality, this kind of conversion should not happen that often (this could be an indicator of a bad/immature interface).
I barely remember all these methods and an extra reason if because of the lack of API asymmetry or consistency in Java constructors/factory methods.
For example, since there is Arrays.asList(...) I would expect Lists.asArray(...) but No!. There is mylist.toArray()!?.
The same problem is not gone when using streams but at least the result array/list does not have the pitfalls non-stream approaches have.

To me, it looks like using Streams is the easiest way of doing this kind of tasks since Java 8 is quite common nowadays :)

These are the packages I would be using...
import java.util.*;
import java.util.Arrays;

Creating arrays

Create array of ints (literal)
int[] array = {7, 8, 9};
Create array of ints (non literal ?)
int[] array = new int[3];
array[0] = 7;
array[0] = 8;
array[0] = 9;
Pretty print array:
String pretty = Arrays.toString(array);
System.out.println(pretty);

Creating lists

Create list of ints. Since List can hold only objects primitives must be boxed. In this case they are boxed into Integer.
List<Integer> list = new ArrayList<>();
list.append(7);
list.append(8);
list.append(9);
Pretty print list.

No direct method, we must convert to array first. See below.

Converting array to list

The usual method is Array.asList(...) however there is a catch: the array must contain objects, otherwise it will create a list with a single object.
// Objects:
String[] objectArray = {"A", "B", "C"};
List<String> objectList = new ArrayList(Arrays.asList(objectArray));
One way to convert primitive array to List would be using Streams (Nowadays I hope you are already using at least Java 8 :p):
import java.util.stream.Collectors;
import java.util.stream.Stream;
// Primitives
int[] array = new int[] {7,8,9};
List<Integer> list = Arrays.stream(array).boxed().collect(Collectors.toList()); // 🤗
Also we can convert arrays of objects to lists in the same manner with streams:
// Objects
String[] objectArray = {"A", "B", "C"};
List<String> objectList = Arrays.stream(objectArray).collect(Collectors.toList()); // 🤗

Converting list to array

List<Integer> list = new ArrayList<>();
//Integer[] array = list.toArray(); // Be careful to not to do this! Check https://stackoverflow.com/a/13647072/149008
Integer[] array = list.toArray(new String[0]);
Another way is to use streams. Which is nicer because we get unboxed ints :)
List<Integer> list = new ArrayList<>();
list.add(7);
list.add(8);
list.add(9);
int[] array = list.stream().mapToInt(Integer::intValue).toArray();  // 🤗
Also we can use streams to convert Object list to Object array
List<String> list = new ArrayList<>();
list.add("A");
list.add("B");
list.add("C");
//String[] array = (String [])list3.stream().map(i -> i).toArray(); // Be careful to not to do this
String[] array = list.stream().toArray(String[]::new);  // 🤗
Useful Link: Java Array to List Examples

2017/10/19

Install swift in docker from scratch

Get Docker Image

Pull latest ubuntu image and check
docker pull ubuntu
docker images
Start ubuntu
docker run -i -t ubuntu:16.04 /bin/bash

Setup things inside Ubuntu

Ubuntu image has almost nothing installed. Lets install some stuff # https://stackoverflow.com/questions/28885137/how-to-run-wget-inside-ubuntu-docker-image
# Update apt-get itself
apt-get update

# Install wget to get swift binaries
apt-get install -y wget

# (Optional) Install your prefered text editor to edit the PATH
apt-get install -y emacs

# Install depencencies swift requires
apt-get install -y clang libicu-dev

# (Optional) Install more dependencies other libraries might need (Vapor, Kitura, etc)
apt-get install -y clang libicu-dev libcurl4-openssl-dev libssl-dev

Install swift

Lets go into some directory (create if needed)
cd /home
mkdir swift4
cd swift4
Get the latests binaries and untar. Latest links can be found in swift.org/download/#releases
wget https://swift.org/builds/swift-4.0-release/ubuntu1604/swift-4.0-RELEASE/swift-4.0-RELEASE-ubuntu16.04.tar.gz
tar -xvzf swift-4.0-RELEASE-ubuntu16.04.tar.gz
Add swift to PATH. Append the following two lines to your ~/.profile file
export SWIFT_HOME="/home/swift4/swift-4.0-RELEASE-ubuntu16.04"
export PATH=$PATH:${SWIFT_HOME}/usr/bin
Finally apply your changes
source ~/.profile
Start using swift!
swift --version
Swift version 4.0 (swift-4.0-RELEASE)
Target: x86_64-unknown-linux-gnu

More Links:

How To Use Swift Programming Language In Ubuntu Linux
How to run wget inside Ubuntu Docker image?
Swift.org/download/#releases

2017/09/04

Requests to server with self signed certificate in iOS

This is kind of old topic but information seems to be fussy to me, I couldn't find all the answers in a a single place so here are my notes.... Let's say we have a server which uses https but with a self signed certificate (オレオレ証明書), and your iOS app needs to do requests to it. Since iOS 8, if I well remember, iOS apps don't work with these servers without additional settings.
Official info from Apple suggest creation of a certificate which means changes in server side which is not an option in my case.
Below I setup my app so requests can be done with an insecure certificate only to my server. This should be done only for development/testing purposes and should not be in production

1. Instruct ATS to ignore your server

Edit Info.plist to have this (From Lory Lory in Stackoverflow):
<key>NSAppTransportSecurity</key>
<dict>
    <key>NSExceptionDomains</key>
    <dict>
        <key>self-signed-certificate.server.com</key>
        <dict>
            <!--Include to allow subdomains too-->
            <key>NSIncludesSubdomains</key>
            <true/>
            <!--Include to allow ssl requests with self signed certificates and other insecure stuff-->
            <key>NSTemporaryExceptionAllowsInsecureHTTPLoads</key>
            <true/>
        </dict>
    </dict>
</dict>
Official (and poor) documentation regarding NSAppTransportSecurity can be found in InfoPlistKeyReference - CocoaKeys. Beware that key names are not even correct, see NSExceptionAllowsInsecureHTTPLoads, instead of NSTemporaryExceptionAllowsInsecureHTTPLoads, etc.

2. Make sure challenge is fulfilled in the WKWebView

Implement the navigation delegate. Remember, at least you should have a naive filter like below. Accepting everything is much more dangerous. This should be done only for testing/developing purposes.
func webView(_ webView: WKWebView, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Swift.Void) {
    if challenge.protectionSpace.host == "self-signed-certificate.server.com" {
        let credential = URLCredential(trust: challenge.protectionSpace.serverTrust!)
        completionHandler(.useCredential, credential)
    } else {
        completionHandler(.performDefaultHandling, nil)
    }
}
Now do setup the webview and do the request, everything should be ready for testing/developing now :)
webView = WKWebView()
webView.frame = CGRect(x: 0, y: 0, width: w, height: h)
webView.navigationDelegate = self

let request = URLRequest(url: URL(string: "https://self-signed-certificate.server.com")!)
webView.load(request)

3. Make sure challenge is fulfilled in the NSURLSession

Implement NSURLSessionDelegate method as below:
func urlSession(_ session: URLSession, task: URLSessionTask, didReceive challenge: URLAuthenticationChallenge, completionHandler: @escaping (URLSession.AuthChallengeDisposition, URLCredential?) -> Void) {
     // Pass test server with self signed certificate
    if challenge.protectionSpace.host == "self-signed-certificate.server.com" {
        completionHandler(.useCredential, URLCredential(trust: challenge.protectionSpace.serverTrust!))
    } else {
        completionHandler(.performDefaultHandling, nil)
    }
}

3. Make sure challenge is fulfilled in Alamofire

Alamofire internally uses NSURLSession so implement something very similar as above is ok.
let sessionManager = SessionManager()
sessionManager.delegate.taskDidReceiveChallengeWithCompletion = { (_, _, challenge, completionHandler) -> Void in
    // Pass test server with self signed certificate
    if challenge.protectionSpace.host == "self-signed-certificate.server.com" {
        completionHandler(.useCredential, URLCredential(trust: challenge.protectionSpace.serverTrust!))
    } else {
        completionHandler(.performDefaultHandling, nil)
    }
}
Now simply do the call (Alamofire example)
sessionManager.request("https://self-signed-certificate.server.com", method: .get).response { response in 
 ...
}

2017/08/17

Preventing Alamofire redirect

I want to NOT follow the redirect, I want to simply stop the redirect and maybe get some info from the response.

Curl request

Plain request to google.com redirects to google.jp (since I am in Japan)
curl -v http://google.com
...
> GET / HTTP/1.1
> Host: google.com
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 302 Found
< Cache-Control: private
< Content-Type: text/html; charset=UTF-8
< Referrer-Policy: no-referrer
< Location: http://www.google.co.jp/?gfe_rd=cr&ei=iFSUWenOIajEXun8iKAN
< Content-Length: 259
< Date: Wed, 16 Aug 2017 14:19:52 GMT
< 
<HTML><HEAD><meta http-equiv="content-type" content="text/html;charset=utf-8">
...
</HTML>
* Connection #0 to host google.com left intact

Alamofire request

// Prevent automatic redirects
let delegate = Alamofire.SessionManager.default.delegate
delegate.taskWillPerformHTTPRedirection = { (_, _, _, _) -> URLRequest? in
    return nil
}

Alamofire.request("http://google.com", method: .get, headers: [:])
    .validate(statusCode: 302..<303)
    .response { (res: DefaultDataResponse) in
        
        // Make sure "Location" header is present and its url is parsed correctly
        guard let response = res.response, 
            let location = response.allHeaderFields["Location"] as? String,
            let url = URL(string: location),
            let components = URLComponents(url: url, resolvingAgainstBaseURL: false) else { 
            return
        }

        // Make sure we get the query parameter we are interested in
        guard let ei = components.queryItems?.first(where: {
            return $0.name == "ei" && $0.value != nil && !($0.value!.isEmpty)
        })?.value else {
            return
        }

        // Voilà
        print("ei query parameter: \(ei)") // iFSUWenOIajEXun8iKAN
}

I skipped the proper error handling, but I think this shows the minimal code required for handling 302 responses.

To restore to original behaviour

Simply assign nil to the handler:
// Restore to original behaviour
let delegate = Alamofire.SessionManager.default.delegate
delegate.taskWillPerformHTTPRedirection = nil
I hope it helps :]

2017/04/29

Compressing and Decompressing NSData

Since iOS9 Compression.framework which we can use for (de)compressing certain chunks of data.

Note: This framework (and below code) is not for directly zipping/unzipping files. The zip format is for compresssing various files and/or directories which require more metadata. The structure of a PKZip file.

Below code simply shows how to compress a particular chunk of NSData.

import Compression

extension Data {
    func compressedData(algorithm: compression_algorithm) -> Data? {
        return performEncoding(compress: true, algorithm: algorithm)
    }

    func decompressedData(algorithm: compression_algorithm) -> Data? {
        return performEncoding(compress: false, algorithm: algorithm)
    }

    private var unsafePointerUInt8: UnsafePointer<UInt8> {
        // based on http://stackoverflow.com/questions/25554986/
        let buffer = UnsafeMutablePointer<UInt8>.allocate(capacity: count)
        let stream = OutputStream(toBuffer: buffer, capacity: count)
        stream.open()
        withUnsafeBytes({ (p: UnsafePointer<UInt8>) -> Void in
            stream.write(p, maxLength: self.count)
        })
        stream.close()
        return UnsafePointer<UInt8>(buffer)
    }
    
    private func performEncoding(compress: Bool, algorithm: compression_algorithm) -> Data? {
        // based on https://gist.github.com/daltheman/4716ec10d6d0f71aba56
        let sourceBuffer = unsafePointerUInt8
        let sourceBufferSize = data.count
        
        let destinationBuffer = UnsafeMutablePointer.allocate(capacity: sourceBufferSize)
        let destinationBufferSize: Int = sourceBufferSize
        
        let status: Int
        if compress {
            status = compression_encode_buffer(
                destinationBuffer, destinationBufferSize,
                sourceBuffer, sourceBufferSize,
                nil, algorithm)
        } else {
            status = compression_decode_buffer(
                destinationBuffer, destinationBufferSize,
                sourceBuffer, sourceBufferSize,
                nil, algorithm)
        }
        
        if status == 0 {
            print("Error with status: \(status)")
            return nil
        }
        print("Original size: \(sourceBufferSize) | Compressed size: \(status)")
        return Data(bytesNoCopy: destinationBuffer, count: status, deallocator: .free)
    }
}

2017/04/14

Add DB2 to maven project

Unfortunately adding below dependencies to pom.xml is not enough for db2 since there is no public maven repository for it.
<dependency>
    <groupId>com.ibm.db2.jcc</groupId>
    <artifactId>db2jcc4</artifactId>
    <version>10.1</version>
</dependency>
<dependency>
    <groupId>com.ibm.db2</groupId>
    <artifactId>db2jcc_license_cu</artifactId>
    <version>10.5</version>
</dependency>
We need to add the jar files to our local maven repository. So if my directory structure looks like this:
.
├── MyMavenProject
│   ├── pom.xml
│   └── src
└── db2-connector
    ├── db2jcc4.jar
    └── db2jcc_license_cu.jar
Then I can do:
cd MyMavenProject

mvn install:install-file -Dfile=../db2-connector/db2jcc4.jar -DgroupId=com.ibm.db2.jcc -DartifactId=db2jcc4 -Dversion=10.1 -Dpackaging=jar -DgeneratePom=true -DcreateChecksum=true

mvn install:install-file -Dfile=db2jcc_license_cu.jar -DgroupId=com.ibm.db2 -DartifactId=db2jcc_license_cu -Dversion=10.5 -Dpackaging=jar -DgeneratePom=true -DcreateChecksum=true

Done! For Reference this is what I always have in my pom.xml so I remember what to do :)
<!-- Install previously: -->
<!-- mvn install:install-file -Dfile=../db2-connector/db2jcc4.jar -DgroupId=com.ibm.db2.jcc -DartifactId=db2jcc4 -Dversion=10.1 -Dpackaging=jar -DgeneratePom=true -DcreateChecksum=true -->
<dependency>
    <groupId>com.ibm.db2.jcc</groupId>
    <artifactId>db2jcc4</artifactId>
    <version>10.1</version>
</dependency>
<!-- Install previously: -->
<!-- mvn install:install-file -Dfile=db2jcc_license_cu.jar -DgroupId=com.ibm.db2 -DartifactId=db2jcc_license_cu -Dversion=10.5 -Dpackaging=jar -DgeneratePom=true -DcreateChecksum=true -->
<dependency>
    <groupId>com.ibm.db2</groupId>
    <artifactId>db2jcc_license_cu</artifactId>
    <version>10.5</version>
</dependency>

2017/04/13

Some DB2 commands and queries

Commands

Switch to db2inst1 user

su - db2inst1

Start and stop DB. All db2 commands should be between these two

db2start

db2 ...

db2stop

List all databases in the instance

db2 LIST DATABASE DIRECTORY

Connect to a certain database

db2 connect to MY_DATABASE_NAME

List all schemas

select schemaname from syscat.schemata

List all tables in all schemas

db2 list tables for all

List tables in a certain schema

db2 list tables for schema DB2INST1

Do a SQL Query

db2 "select * from MY_TABLE"

Queries

Datetime vs Timestamp. It looks timestamp is better, at least it was a better fit in all cases I have used it so far. Nir's Answer: Should I use field 'datetime' or 'timestamp'?
CREATE TABLE mytable (id INTEGER, made_on TIMESTAMP)
INSERT INTO mytable (id, made_on) VALUES ( 1, TIMESTAMP '2005-05-13 07:15:31.123456789')
INSERT INTO mytable (id, made_on) VALUES ( 2, TIMESTAMP '2005-05-13 07:15:31')
INSERT INTO mytable (id, made_on) VALUES ( 3, DATE '2005-05-13')
SELECT * FROM mytable;
ID          MADE_ON                   
----------- --------------------------
          1 2005-05-13-07.15.31.123456
          2 2005-05-13-07.15.31.000000
          3 2005-05-13-00.00.00.000000

Sequences

Drop and create a sequence

DROP SEQUENCE CUSTOMER_NUMBER_SEQ RESTRICT;
CREATE SEQUENCE CUSTOMER_NUMBER_SEQ START WITH 1 INCREMENT BY 1 MINVALUE 0 NO MAXVALUE CYCLE CACHE 4;

Select the next one

SELECT NEXT VALUE FOR CUSTOMER_NUMBER_SEQ FROM SYSIBM.SYSDUMMY1;
1

Select next one with padding zeros

SELECT LPAD(NEXT VALUE FOR CUSTOMER_NUMBER_SEQ, 8, '0') FROM SYSIBM.SYSDUMMY1;
00000002

Select previous one (most recently generated)

SELECT PREVIOUS VALUE FOR CUSTOMER_NUMBER_SEQ from SYSIBM.SYSDUMMY1;
2

Primary keys and Foreign keys

Create my users table. Make primary from definition.
DROP TABLE "T_USER";
CREATE TABLE "T_USER"(
 "USER_ID" VARCHAR(16) NOT NULL PRIMARY KEY,
 "PASSWORD" VARCHAR(32) NOT NULL,
 "FULLNAME" VARCHAR(20) NULL,
 "PRIVILEGE" INT NOT NULL,
 "CREATE_TIME" TIMESTAMP NULL DEFAULT CURRENT_TIMESTAMP,
 "UPDATE_TIME" TIMESTAMP NULL
);

CREATE UNIQUE INDEX "USERNAME_UNIQUE" ON "T_USER" ("USER_ID");
COMMENT ON COLUMN T_USER.PRIVILEGE IS 'PRIVILEGE 0:Admin, 1:Editor, 2:Regular';
Create projects table. Make primary key by altering the table.
CREATE TABLE "T_PROJECT" (
 "PROJECT_CODE" VARCHAR(5) NOT NULL,
 "PROJECT_NAME" VARCHAR(60)
);

CREATE UNIQUE INDEX "T_PROJECT_PK" ON "T_PROJECT" ("PROJECT_CODE");
ALTER TABLE "T_PROJECT" ADD CONSTRAINT "T_PROJECT_PK" PRIMARY KEY ("PROJECT_CODE");
Finally create a mapping table, with two foreign keys:
DROP TABLE "T_USER_PROJECT";
CREATE TABLE "T_USER_PROJECT" (
 "USER_ID" VARCHAR(16) NOT NULL,
 "PROJECT_CODE" VARCHAR(5) NOT NULL,
 CONSTRAINT "USER_ID_FK" FOREIGN KEY (USER_ID)
  REFERENCES T_USER (USER_ID) ON DELETE RESTRICT,
 CONSTRAINT "PROJECT_CODE_FK" FOREIGN KEY (PROJECT_CODE)
  REFERENCES T_PROJECT (PROJECT_CODE) ON DELETE RESTRICT
);
This is how the tables relate:

Insert some data:
INSERT INTO T_USER (USER_ID,PASSWORD,FULLNAME,PRIVILEGE) VALUES
 ('admin',  'admin',  'Admin',  0),
 ('editor', 'editor', 'Editor', 1),
 ('user',   'user',   'User',   2);

INSERT INTO T_PROJECT (PROJECT_CODE,PROJECT_NAME) VALUES
 ('58000','Project0'),
 ('58001','Project1');

INSERT INTO T_USER_PROJECT (USER_ID, PROJECT_CODE) VALUES
 ('admin',  '58000'),
 ('admin',  '58001'),
 ('editor', '58000'),
 ('user',   '58001');
Inner join:
SELECT 
 P.PROJECT_CODE AS projectCode,
 P.PROJECT_NAME as projectName,
 M.USER_ID AS userId,
 U.PRIVILEGE AS privilege,
 CASE WHEN (U.PRIVILEGE = 0) THEN 'Y' ELSE 'N' END as admin,
 CASE WHEN (U.PRIVILEGE = 1 OR U.PRIVILEGE = 0) THEN 'Y' ELSE 'N' END as editor
FROM T_PROJECT P
INNER JOIN T_USER_PROJECT M ON M.PROJECT_CODE = P.PROJECT_CODE
INNER JOIN T_USER U ON M.USER_ID = U.USER_ID
AND M.USER_ID = 'admin'
AND M.PROJECT_CODE = '58000';

See more:

2017/03/12

MapView for the busy

I has been a while since I don't do iOS. These are my notes, hopefully helpful for others.

Just show me code

Here you go. Full working example :)

Show a map

Very simple, create a view controller with a MapKit MapView object and you are done. A MapView without delegate is still usable however I set it (in interface builder) for future use.
import UIKit
import MapKit

class MapViewController: UIViewController, MKMapViewDelegate {
    @IBOutlet fileprivate weak var mapView: MKMapView!
}

Get and show current location

Get current location is a task of CoreLocation. We need to receive updates from CoreLocationManagerDelegate so render current location in our mapView. Location itself will be stored in fromLocation and fromLocationAnnotation will be used to represent a pin at such location.
import UIKit
import MapKit
import CoreLocation

class MapInfoDetailViewController: UIViewController, MKMapViewDelegate, CLLocationManagerDelegate {
    @IBOutlet fileprivate weak var mapView: MKMapView!
    fileprivate var locationManager: CLLocationManager?
    fileprivate var fromLocation: CLLocation?
    fileprivate var fromLocationAnnotation: MKPointAnnotation?
}
We start the location manager
override func viewDidLoad() {
    super.viewDidLoad()

    // Start location manager
    if CLLocationManager.locationServicesEnabled() {
        locationManager = CLLocationManager()
        locationManager?.delegate = self
        locationManager?.desiredAccuracy = kCLLocationAccuracyBest
        locationManager?.requestAlwaysAuthorization()
        locationManager?.startUpdatingLocation()
    }
}
Note that in recent versions of iOS we need to add the following to the Info.plist otherwise CoreLocation will not work
<key>NSLocationAlwaysUsageDescription</key>
<string>🍌Short explanation of why you will require location services here🍌</string>
Here is the method that receives locations updates. This method is called several times in very short intervals
func locationManager(_ manager: CLLocationManager, didUpdateLocations locations: [CLLocation]) {
    guard let location = locations.last else { return }

    fromLocation = location

    if fromLocationAnnotation == nil {
        // First time: Create and add annotation
        fromLocationAnnotation = MKPointAnnotation()
        fromLocationAnnotation!.coordinate = fromLocation!.coordinate
        fromLocationAnnotation!.title = "Current Location"
        mapView.addAnnotation(fromLocationAnnotation!)
    } else {
        // Not first time: Update annotation
        mapView.removeAnnotation(fromLocationAnnotation!)
        fromLocationAnnotation!.coordinate = fromLocation!.coordinate
        mapView.addAnnotation(fromLocationAnnotation!)
    }
    mapView.showAnnotations(mapView.annotations, animated: true)

    if location.horizontalAccuracy < 30 {
        // IMO 30m is accurate enough to turn off location services to save battery :)
        locationManager?.stopUpdatingLocation()
    }
}
Previous method adds an MKPointAnnotation to the mapView. MKMapView will render the default annotation view for the given annotation. So far we our current location :)

Customize the pin

To customize the view we need to implement MKMapViewDelegate method. When we add MKAnnotation (usually a MKPointAnnotation) the map view will consult its delegate to see if there is a view for the given annotation. If we return nil or do not implement the delegate method is will draw the default view (a red pin). Usually you want to create a view with some extra information special to the location (For example number of likes for that place, an explanation of the place, etc) and a way to pass data from your model to the view is via a subclass of MKAnnotation. Each MKAnnotationView object will have a reference to an MKAnnotation so we have put data here.
import MapKit

class PlaceAnnotation: MKPointAnnotation {
    var label: String?
}
To use it instead of creating MKPointAnnotation we use our own:
...
// Instead of:
// fromLocationAnnotation = MKPointAnnotation()
// We can create our MKAnnotation and pass any data we might need later
let annotation = PlaceAnnotation()
annotation.label = "出発"
fromLocationAnnotation = annotation
...
The delegate method:
public func mapView(_ mapView: MKMapView, viewFor annotation: MKAnnotation) -> MKAnnotationView? {
    guard let annotation = annotation as? PlaceAnnotation else {
        // Other annotations will show the default pin
        return nil
    }
    // Place annotation
    var annotationView: MKAnnotationView?
    let reuseId = String(describing: PlaceAnnotation.self)
    annotationView = mapView.dequeueReusableAnnotationView(withIdentifier: reuseId)
    if annotationView == nil {
        // Create annotation view
        annotationView = MKAnnotationView(
            annotation: annotation,
            reuseIdentifier: reuseId)
        annotationView?.canShowCallout = true
        annotationView?.image = image(text: annotation.label)
    } else {
        // Update annotation view (update the least possible)
        annotationView?.annotation = annotation
        annotationView?.image = image(text: annotation.label)
    }
    return annotationView
}
I am using a regular MKAnnotationView and simply customizing the image according the given data (I am creating an UIImage on the fly). In my small app I have just a few annotations so this does not cost me anything. If performance becomes a problem then we should create subclass MKAnnotationView and render things there rather than setting a different UIImage every time, to really reuse it. If you are curious about image(text: annotation.label) check code here.


Get and show location from an address

The act of find coordinates from a address text is called Geo Coding. Happily there is a geocoder class in CoreLocation so lets use it:

fileprivate var toLocation: CLLocation?

...

func getLocationAndShowRoute(address: String) {
    let geocoder = CLGeocoder()
    geocoder.geocodeAddressString(address) { (placemarks, error) in
        if let error = error {
            print("Geocoder error. " + error.localizedDescription)
            return
        }
        guard let placemark = placemarks?.last else {
            print("Geocoder error. No placemarks found")
            return
        }
        guard let location = placemark.location else {
            print("Geocoder error. No location in placemark")
            return
        }
        // Add location
        let annotation = MKPointAnnotation()
        annotation.coordinate = location.coordinate
        annotation.title = address
        self.mapView.addAnnotation(annotation)
        self.toLocation = location
    }
}

Draw route between two locations

By now we should have two locations:
  • fromLocation: current location found with help of CLLocationManager
  • toLocation: an arbitrary location found by geocoding an address

func showRoute(transportType: MKDirectionsTransportType) {
    guard let from = fromLocation else {
        print("showRoute: no fromLocation")
        return
    }
    guard let to = toLocation else {
        print("showRoute: no toLocation")
        return
    }
    
    // Search routes in MapKit (Japanese article)
    // http://qiita.com/oggata/items/18ce281d5818269c7281
    let fromPlacemark = MKPlacemark(coordinate: from.coordinate, addressDictionary: nil)
    let toPlacemark = MKPlacemark(coordinate: to.coordinate, addressDictionary: nil)
    
    let fromItem = MKMapItem(placemark:fromPlacemark)
    let toItem = MKMapItem(placemark:toPlacemark)
    
    let request = MKDirectionsRequest()
    request.source = fromItem
    request.destination = toItem
    request.requestsAlternateRoutes = false // only one route
    request.transportType = transportType
    
    let directions = MKDirections(request:request)
    directions.calculate { response, error in
        if let error = error {
            print("Route search error. " + error.localizedDescription)
            return
        }
        guard let route = response?.routes.last else {
            print("Route search error. No routes found")
            return
        }
        self.mapView.removeOverlays(self.mapView.overlays)
        self.mapView.add(route.polyline)
    }
}
Tadaa!
Full code available here.

2016/09/24

Generate PDF documentation from swagger annotations

TL DR;: Check this working example github.com/nacho4d/rest_to_pdf_example

Overview

I have a REST (JAX-RS) Java Web App and I use Maven (pom.xml) to build it.
  1. Annotate: Use swagger-annotations to annotate my endpoints.
  2. Generate swagger file: Use the swagger-maven-plugin to generate swagger file from sources
  3. Generate asciidoc file: Use the swagger2markup-maven-plugin to generate asciidoc from swagger file
  4. Generate pdf file: Use asciidoctor-maven-plugin to generate PDF from asciidoc file

0. Preparation

I use a couple of variables: swagger.directory and asciidoctor.directory.
<properties>
    <swagger.directory>${project.build.directory}/generated-docs/swagger</swagger.directory>
    <asciidoctor.directory>${project.build.directory}/generated-docs/asciidoc</asciidoctor.directory>
</properties>
One of the dependencies (markdown_to_asciidoc) of swagger2markup-maven-plugin need below repository:
<pluginRepositories>
    <pluginRepository>
        <id>jcenter-snapshots</id>
        <name>jcenter</name>
        <url>http://oss.jfrog.org/artifactory/oss-snapshot-local/</url>
    </pluginRepository>
    <pluginRepository>
        <snapshots>
            <enabled>false</enabled>
        </snapshots>
        <id>jcenter-releases</id>
        <name>jcenter</name>
        <url>http://jcenter.bintray.com</url>
    </pluginRepository>
</pluginRepositories>

1. Annotate

Add io.swagger dependency inside dependenciesand start annotating. Basically just add @ApiOperation(value = "Description of endpoint") to each endpoint (or swagger file will contain 0 endpoints). Near @POST, @GET, @DELETE, etc should be fine. See a more detailed example.
<dependency>
    <groupId>io.swagger</groupId>
    <artifactId>swagger-annotations</artifactId>
    <version>1.5.10</version>
</dependency>

2. Generate swagger file

This looks a bit lengthy. but IMO should work for most people. Please adjust locations to have a list of all packages where your endpoints classes are (or swagger file will contain 0 endpoints). This should be inside build/plugins
<plugin>
    <groupId>com.github.kongchen</groupId>
    <artifactId>swagger-maven-plugin</artifactId>
    <version>3.1.4</version>
    <configuration>
        <apiSources>
            <apiSource>
                <springmvc>false</springmvc>
                <locations>
                    <location>com.company.com.support.service</location>  <!-- Modify for your project -->
                </locations>
                <schemes>http,https</schemes>
                <host>my.host.net</host>
                <basePath>/api</basePath>
                <info>
                    <title>Some System - API Reference</title>
                    <version>v1</version>
                    <description>API Reference Description.</description>
                    <termsOfService>TOS</termsOfService>
                    <contact>
                        <email>me@email.com</email>
                        <name>Me</name>
                        <url>www.company.com</url>
                    </contact>
                    <license>
                        <url>http://www.apache.org/licenses/LICENSE-2.0.html</url>
                        <name>Apache 2.0</name>
                    </license>
                </info>
                <outputPath>${swagger.directory}/document.html</outputPath>
                <swaggerDirectory>${swagger.directory}</swaggerDirectory>
                <outputFormats>yaml</outputFormats>
            </apiSource>
        </apiSources>
    </configuration>
    <executions>
        <execution>
            <?m2e execute onConfiguration?>
            <phase>compile</phase>
            <goals>
                <goal>generate</goal>
            </goals>
        </execution>
    </executions>
</plugin>
I have attached generate goal of this plugin to the compile phase. Hence we should be able to generate target/generated-docs/swagger/swagger.yaml file with:
mvn compile

2. Generate asciidoc file

I set the input to be the output of the previous plugin. Also set the output file (not output directory) so everything is concatenated in a single file. Place this inside build/plugins after swagger-maven-plugin.
<plugin>
    <groupId>io.github.swagger2markup</groupId>
    <artifactId>swagger2markup-maven-plugin</artifactId>
    <version>1.0.1</version>
    <configuration>
        <swaggerInput>${swagger.directory}/swagger.yaml</swaggerInput>
        <outputFile>${asciidoctor.directory}/ApiDocumentation</outputFile>
        <config>
            <swagger2markup.markupLanguage>ASCIIDOC</swagger2markup.markupLanguage>
        </config>
    </configuration>
    <executions>
        <execution>
            <phase>prepare-package</phase>
            <goals>
                <goal>convertSwagger2markup</goal>
            </goals>
        </execution>
    </executions>
</plugin>
I have attached convertSwagger2markup goal of this plugin to the prepare-package phase. Hence we should be able to generate target/generated-docs/asciidocs/ApiDocumentation.adoc file with:
mvn package

3. Generate pdf file

Again, I set the input to be the output of the previous plugin. Place this inside build/plugins after swagger2markup-maven-plugin.
<plugin>
    <groupId>org.asciidoctor</groupId>
    <artifactId>asciidoctor-maven-plugin</artifactId>
    <version>1.5.3</version>
    <dependencies>
        <dependency>
            <groupId>org.asciidoctor</groupId>
            <artifactId>asciidoctorj-pdf</artifactId>
            <version>1.5.0-alpha.11</version>
        </dependency>
    </dependencies>
    <configuration>
        <sourceDirectory>${asciidoctor.directory}</sourceDirectory>
        <attributes>
            <sourcedir>${project.build.sourceDirectory}</sourcedir>
        </attributes>
    </configuration>
    <executions>
        <execution>
            <id>generate-pdf-doc</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>process-asciidoc</goal>
            </goals>
            <configuration>
                <backend>pdf</backend>
                <sourceHighlighter>rouge</sourceHighlighter>
                <attributes>
                    <icons>font</icons>
                    <pagenums/>
                    <toc/>
                    <idprefix/>
                    <idseparator>-</idseparator>
                    <sectnums>true</sectnums>
                </attributes>
            </configuration>
        </execution>
    </executions>
</plugin>
Similarly process-asciidoc goal of this plugin to the prepare-package phase. Hence we should be able to generate target/generated-docs/ApiDocumentation.pdf file with:
mvn package
That is all folks :)
PDF自動生成できた!APIリファレンスと頼まれて「えええええーーーーー」と言わなくなったはず、(笑)

Links

2016/09/16

flex-box

Recently I use flex-box a lot. Flex-box is kind of UIStackView in iOS (Flex-box spec was first than UIStackView). I wrote some samples so I don't forget specially about flex-grow, flex-shrink and flex-basis
JSBin - Flexbox examples - https://jsbin.com/kosajor
-- Sorry I don't have a PRO account so embedded code is not available :(

2016/09/04

Compiling with Java 8 in Bluemix

The devops service of Bluemix is nice but at this moment it uses java 7 by default. In order to compile to Java 8 we should modify the pom.xml so maven will try to use java8 when compiling (sames as this post in Japanese):
<project >
    ...

    <properties>
        ...
        <java.version>1.8</java.version>
    </properties>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.1</version>
                <configuration>
                    <source>${java.version}</source>
                    <target>${java.version}</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>
We should also modify the configuration of the build stage in the pipeline. We need to set JAVA_HOME so maven can find it (At first I tried to do set JBP_CONFIG_IBMJDK as adviced via the manifest.yaml but it didn't work).
# Set Java8
export JAVA_HOME=~/java8
export PATH="$JAVA_HOME/bin:$PATH"
Now we are ready to go :)

2016/08/27

"Password is not encoded" on WAS Liberty server.xml

There are some features in WAS Liberty Server (like apiDiscovery-1.0) that require stuff (like keyStore and basicRegistry) that will require passwords. If those passwords are note encoded we get warnings. It is was not clear to me how to encode a password. These are my notes :)

Encoding a password

We need the securityUtility command from bin directory of the server installation directory. And simply use encode subcommand. Since I am using the default key store defaultKeyStore everything is very simple:
# This is the location eclipse says my server is
cd /Users/ignacio/Documents/ibm/was16.0.0.2/usr
# Go to the bin directory
cd ../bin/

# encode all my passwords
./securityUtility encode keyStorePwd
{xor}NDomDCswLToPKDs=
./securityUtility encode admin
{xor}PjsyNjE=
./securityUtility encode member
{xor}MjoyPTot

Update the server.xml

Now, instead of passwords in plain text we copy and paste above results. (See my full server.xml)
<server>

    ...

    <keyStore id="defaultKeyStore" password="{xor}NDomDCswLToPKDs=" /> <!-- password="keyStorePwd" -->

    <basicRegistry>
        <user name="admin" password="{xor}PjsyNjE"/>
        <user name="member" password="{xor}MjoyPTot"/>
        <group name="admins">
            <member name="admin"/>
        </group>
        <group name="member">
            <member name="member"/>
        </group>
    </basicRegistry>

    ...

</server>
voilà! no warnings -- So much better :)

PS: defaultKeyStore 's password must be "keyStorePwd" other password didn't work for me. I can't remember where exactly I got it but it was a site from IBM documentation somewhere in the past.

Hold on, I don't know where is my server installation directory!
In case you installed via eclipse and forgot where it was installed, go the server properties and check the location.

More references

2016/08/24

Remove unnecessary files from java web application war file

I usually use Visual Code for coding in javascript and other client-side development. That means that I usually have .vscode/launch.json, .vscode/launch.json and jsconfig.json which are good for development but not needed in the final war file (created when doing mvn -B package)
src/main/webapp/
├── .vscode/
│   ├── launch.json
│   └── settings.json
├── WEB-INF/
├── css/
│   └── ...
├── images/
│   └── ...
├── index.html
├── js/
│   └── ...
└── jsconfig.json
Below piece should be added in the pom.xml to tell maven to exclude certain files. When the start (*) is not enough we can use %regex[ ]. More details: Maven war plugin - including/excluding files from war.
<project>
  ...
  <build>
    <plugins>

      <plugin>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.6</version>
        <configuration>
          <!-- Exclude configuration files for Visual Code editor -->
          <packagingExcludes>jsconfig.json</packagingExcludes>
          <packagingExcludes>.vscode/*</packagingExcludes>
          <!-- Exclude documentation and other unnecessary files from d3.js -->
          <packagingExcludes>js/vendor/d3/*.md</packagingExcludes>
        </configuration>
      </plugin>

    </plugins>
  </build>
  ...
</project>

2016/08/09

License

I realized all code and stuff I write here does not have a license. I never really thought of such necessity since code I write here are usually just chunks of code, not finished products. But some people might mentioned recently.
So today I am releasing all this work under the BSD Zero Clause License (0BSD).

Basically, it allows unlimited freedom with the software without the requirements to include the copyright notice, license text, or disclaimer in either source or binary forms.
Copyright (C) 2016 by Guillermo Ignacio Enriquez Gutierrez <nacho4d@gmail.com>
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted.

THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
More about the 0BSD: Fun read about 0BSD:

2016/08/03

javax.json.Json めんどくさすぎる

var jsonObject = JSON.parse("{\"key\":\"value\"}");
var jsonArray = JSON.parse("[1,true,\"word\"]");
var jsonString = JSON.stringify({key:"value"});
のjavascriptみたいに一行でString ⇄ JsonObject変換したいけど javax.json.JsonのAPIがめんどくさすぎる。特にJsonWriterとか使わない使いたくない場合。

Lazy Man approach


使い方はjsに少し似ていて lazyなおれにとってはマシかと。。。
JsonObject jsonObject = JsonUtils.parse("{\"key\":\"value\"}");
JsonArray jsonArray = JsonUtils.parseArray("[1,true,\"word\"]");
String jsonString = JsonUtils.stringify(jsonObject);

復習

使い方をいつもわすれるので ちょっとメモを

JsonObjectやJsonArrayの構築

基本的に Json.createObjectBuilder().xyz.build()Json.createArrayBuilder().abc.build()でオブジェクトやアレーを入れっこ作っていく。xyzとabcとはadd(...)関数のこと
例えば、このようなjsonオブジェクトを組み立てたいとしたら
{
  keyString: "String",
  keyBoolean: true,
  keyArraySimple: [
    1,
    11,
    111
  ],
  keyArrayObject: [{
      lang: "ja",
      where: "どこ"
    },{
      lang: "es",
      where: "donde"
  }],
  keyObject: {
    git: "svn",
    github: "sourceforge"
  },
  keyNull: null
}
このように書ける
JsonObject json = Json.createObjectBuilder()
        .add("keyString", "string")
        .add("keyBoolean", true)
        .add("keyArraySimple", Json.createArrayBuilder()
                .add(1)
                .add(11)
                .add(111).build())
        .add("keyArrayObject", Json.createArrayBuilder()
                .add(Json.createObjectBuilder()
                        .add("lang", "ja")
                        .add("where", "どこ").build())
                .add(Json.createObjectBuilder()
                        .add("lang", "es")
                        .add("where", "sourceforge").build()))
        .add("keyObject", Json.createObjectBuilder()
                .add("git", "svn")
                .add("github", "sourceforge").build())
        .addNull("keyNull").build();
注意 JsonObjectJsonArrayはinmutable。mutateできるのはJsonObjectBuilderJsonArrayBuilderの時点までだ。つまり.build()メソッドの直前まで。

型を調べる

例えば JsonArrayJsonObjectの中身が固定ではないとき getHogeHogeして
ClassCastExceptionをキャッチするより get() してから getValueType() を行う方がよさそう.
JsonArray jsonArray = Json.createArrayBuilder()
         .add("some string")
         .add(true)
         .add(1)
         .addNull().build();
for (int i = 0; i < jsonArray.size(); ++i) {
    JsonValue value = jsonArray.get(i);
    switch (value.getValueType()) {
    case STRING:
        String someString = jsonArray.getString(i);
        ...
        break;
    case ARRAY:
        boolean positive = jsonArray.getBoolean(i);
        ...
        break;
    case NULL:
        ...
        break;
    ...
    default:
        ...
        break;
    }
}

HttpServletRequestのbodyをJsonObjectに直接読む

前はJsonReaderやJsonWriterを使いたくないとかぶつぶつ言ったかもしれないけど、やはり使えるならば使った方がよさそう。
リクエストのボディをStringにしてからまたJsonObjectにする方がめんどくさい。一発でreaderから直接JsonObjectしよう。書き出しのときは 一旦Stringにしてからじゃないやりかたを今のところ思いつかない。
public class MyServlet extends HttpServlet {
    @Override 
    protected void doPost(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

        // Only needed when client does not set it properly. 
        // IMO, Ideally this should not be needed
        request.setCharacterEncoding(StandardCharsets.UTF_8.toString());

        // リクエストボディを読込
        JsonReader bodyJsonReader = Json.createReader(new BufferedReader(request.getReader()));
        JsonObject bodyJsonObject = bodyJsonReader.readObject();
        
        ...

        // リスポンスを返却
        String jsonString = JsonUtils.stringify(Json.createObjectBuilder.build());
        response.setCharacterEncoding(StandardCharsets.UTF_8.name());
        response.setContentType("application/json");
        response.getWriter().write(jsonString);
    }
}

2016/08/01

Cookie config in servlet web app

I want to change session cookie name, make it secure and httpOnly. So, apparently this should be done via the web.xml file.
Based in Servlet Programming techniques - cookie-config (Japanese) I edited the src/main/webapp/WEB-INF/web.xml file:
<web-app version="3.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://xmlns.jcp.org/xml/ns/javaee" xsi:schemalocation="http://xmlns.jcp.org/xml/ns/javaee http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd">

    <session-config>
        <cookie-config>
            <name>SASSESSION</name>
            <http-only>true</http-only>
            <secure>true</secure>
        </cookie-config>
    </session-config>

</web-app>
Now we have a customized session cookie:
curl -D - -o /dev/null http://localhost:9080/MyContext
...
X-Powered-By: Servlet/3.1
...
Set-Cookie: SASSESSION=0000GEg4qyTVnFKTdrdFjUC8eK0:5e81c98f-2566-49b3-9d72-b0efc4489ff8; Path=/; Secure; HttpOnly
...

Notes

2016/07/07

Cloudant Analizers

The other day I rewrote CloudanDB (Which is in Japanese 日本語) and realized that some things were not working because of the analizer I was using. So this post is about them.

What are Cloudant Analizers?

From the official documentation:
Analyzers are settings which define how to recognize terms within text. This can be helpful if you need to index multiple languages.
I decided to try some of the analyzers and put my results here. Test text has two completely different languages (English and Japanese) and some other symbols which I think it is enough to find out how different analysers behave.

How to test Analizers

The _search_analyze API is for this purpose
FILE=analyze_classic.json
echo '{"analyzer": "classic","text": "Evening of the seventh"}' > $FILE

# Use _search_analyze
CLOUDANT='https://<username>:<password>@<username>.cloudant.com'
curl -X POST ${CLOUDANT}/_search_analyze  -H 'Content-Type: application/json' --data @${FILE}
{"tokens":["evening","seventh"]}

Experiment

Today is a Japanese festival Tanabata so I will use a piece of this romantic song I like very much Suspicious Mind - Elvis Presley and a part of its japanese translation I found here
We're caught in a trap\n
I can't walk out\n
Because I love you too much baby♬.\n
Sent by nacho@email.com at 2016-07-07 18:00:29 +0900 ★\n
はまった罠から\n
出られないんだ\n
ほんとに君に首ったけなんだ♬.\n
2016年7月7日18時0分29秒にnacho@email.jpより ★
I added new lines \n, ♬★ marks, an email and a timestamp to make it more interesting ;-]

Results

AnalyzerResult tokensLength
classic["we're", "caught", "trap", "i", "can't", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "nacho@email.com", "2016-07-07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nacho@email.jp", "よ", "り"]64
email["we're", "caught", "trap", "i", "can't", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "nacho@email.com", "2016", "07", "07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nacho@email.jp", "よ", "り"]66
keyword["We're caught in a trap\nI can't walk out\nBecause I love you too much baby♬.\nSent by nacho@email.com at 2016-07-07 18:00:29 +0900 ★\nはまった罠から\n出られないんだ\nほんとに君に首ったけなんだ♬.\n2016年7月7日18時0分29秒にnacho@email.jpより ★"]1
simple["we", "re", "caught", "in", "a", "trap", "i", "can", "t", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "by", "nacho", "email", "com", "at", "はまった罠から", "出られないんだ", "ほんとに君に首ったけなんだ", "年", "月", "日", "時", "分", "秒にnacho", "email", "jpより"]35
standard["we're", "caught", "trap", "i", "can't", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "nacho", "email.com", "2016", "07", "07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nacho", "email.jp", "よ", "り"]68
whitespace["We're", "caught", "in", "a", "trap", "I", "can't", "walk", "out", "Because", "I", "love", "you", "too", "much", "baby♬.", "Sent", "by", "nacho@email.com", "at", "2016-07-07", "18:00:29", "+0900", "★", "はまった罠から", "出られないんだ", "ほんとに君に首ったけなんだ♬.", "2016年7月7日18時0分29秒にnacho@email.jpより", "★"]29
english["we'r", "caught", "trap", "i", "can't", "walk", "out", "becaus", "i", "love", "you", "too", "much", "babi", "sent", "nacho", "email.com", "2016", "07", "07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nacho", "email.jp", "よ", "り"]68
spanish["we'r", "caught", "in", "trap", "i", "can't", "walk", "out", "becaus", "i", "love", "you", "too", "much", "baby", "sent", "by", "nach", "email.com", "at", "2016", "07", "07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nach", "email.jp", "よ", "り"]71
japanese["we", "re", "caught", "in", "a", "trap", "i", "can", "t", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "by", "nacho", "email", "com", "at", "2016", "07", "07", "18", "00", "29", "0900", "はまる", "罠", "出る", "ほんとに", "君", "首ったけ", "2", "0", "1", "6", "年", "7月", "7", "日", "1", "8", "時", "0", "分", "2", "9", "秒", "nacho", "email", "jp"]56
cjk["we're", "caught", "trap", "i", "can't", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "nacho", "email.com", "2016", "07", "07", "18", "00", "29", "0900", "はま", "まっ", "った", "た罠", "罠か", "から", "出ら", "られ", "れな", "ない", "いん", "んだ", "ほん", "んと", "とに", "に君", "君に", "に首", "首っ", "った", "たけ", "けな", "なん", "んだ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒に", "nacho", "email.jp", "より"]63
arabic["we're", "caught", "in", "a", "trap", "i", "can't", "walk", "out", "because", "i", "love", "you", "too", "much", "baby", "sent", "by", "nacho", "email.com", "at", "2016", "07", "07", "18", "00", "29", "0900", "は", "ま", "っ", "た", "罠", "か", "ら", "出", "ら", "れ", "な", "い", "ん", "だ", "ほ", "ん", "と", "に", "君", "に", "首", "っ", "た", "け", "な", "ん", "だ", "2016", "年", "7", "月", "7", "日", "18", "時", "0", "分", "29", "秒", "に", "nacho", "email.jp", "よ", "り"]72

Commentaries

Some interesting things to note:
  • The only notorious difference between standard and classic is the treat of "2016-07-07" and emails. standard got the emails wrong.
  • email looks like standard but with the emails right
  • whitespace looks like a good option when text has symbols (ie.: ♬, ★). There are still there as tokens!
  • keyword just one token. I think it would be useful for exact match searches
  • None of the analyzers, not even keyword preserved the new line character \n.
  • Several occurrences of the same string are valid. (ie.: "i" and "★")
  • I find funny "we're" became "we'r" when english was used. Also "baby" became "babi".
  • Japanese words were tokenized as characters in english. Makes really hard (if possible) to do a useful search using Japanese
  • I am depressed that spanish didn't get my name "nacho" right. For some reason it became "nach". I guess it is trying to get the root of it, since there is also nacha, nachito, nachita, nachos, nachas, etc.
  • cjk (chinese/japanese/korean) complete broke the japanese words. I know it is hard to parse Japanese but cjk is not helping at all here.
  • I have no idea why I tried arabic I literally have 0 knowledge of the language to comment something about it.


More on "nacho"
curl -X POST ${CLOUDANT}/_search_analyze  -H 'Content-Type: application/json' -d '{"analyzer":"spanish", "text":"nacho"}'
{"tokens":["nach"]}
curl -X POST ${CLOUDANT}/_search_analyze  -H 'Content-Type: application/json' -d '{"analyzer":"spanish", "text":"nachos"}'
{"tokens":["nach"]}
curl -X POST ${CLOUDANT}/_search_analyze  -H 'Content-Type: application/json' -d '{"analyzer":"spanish", "text":"Nachos"}'
{"tokens":["nach"]}
🇵🇪 Creo que voy a reclamar a los de Cloudant o Apache para que mi nombre salga bien! Ahorita mismo!!🇵🇪 (O sera que esta sacando la raiz de la palabra?)

2016/07/04

Cordova application skeleton

Create application

Create it using the cordova command. We can get more help with: cordova create -help.

# cordova create <PATH> [ID [NAME [CONFIG]]] [options]
cordova create hello com.example.hello HelloWorld
cd hello

Next, we add platforms. Here I only add iOS and Android but there are a lot more.

cordova platform add ios --save
cordova platform add android --save

Working with git

I am not really into Cordova applications but as any other platforms I tend to have my repositories the smallest possible. There should be a command that downloads all dependencies so all users are able to have the same environment in the end. For such reason I ignore everything in platforms/ and plugins/ directories

My .gitignore:

platforms/
plugins/

The command that will make my environment ready will be:

cordova prepare

It copies files for specified platforms, or all platforms, so that the project is ready to build with each SDK.

So far our file structure will look like this:

.
├── config.xml
├── hooks/
│   └── README.md
├── platforms/
│   ├── android/
│   ├── ios/
│   └── platforms.json
├── plugins/
│   ├── android.json
│   ├── cordova-plugin-whitelist/
│   ├── fetch.json
│   └── ios.json
└── www/
    ├── css/
    ├── img/
    ├── index.html
    └── js/

Running the app

On iOS

Install ios-sim to be able to lunch the simulator. This is a one time task.

npm install -g ios-sim

Build and run

cordova build ios # build Xcode project
cordova emulate ios # launch simulator

Using Xcode

Command cordova build ios creates an Xcode project. This is quite useful specially for debbuging plugins.

 open -a Xcode platforms/ios/HelloWorld.xcodeproj/

On Android

Build and run

cordova build android
cordova run android # Run on connected device
cordova emulate android # Run on simulator

Using Android studio

Command cordova build android, creates an Android Studio project. Again, this is quite useful for debbuging plugins.

open -a "Android Studio" platforms/android/

2016/06/28

Table of contents generator in javascript

There are a lot of approaches arround. Lets add one more to the pile:

Javascript

No need of jquery or other stuff. Just plain and simple vanilla js ;]

function TableOfContents(tocId, containerId) {
    var toc = document.getElementById(tocId);
    toc.innerHTML = "";
    toc.className += " toc";
    var container = containerId ? document.getElementById(containerId) : toc.parentElement;
    var headings = [].slice.call(container.querySelectorAll('h1, h2, h3, h4, h5, h6'));
    var closeLevel = function(e, levels) {
        for (var i = 0; i < levels && e.parentElement && e.parentElement.parentElement; i++) {
            e = e.parentElement.parentElement;
        }
        return e;
    };
    var createLiWithAnchor = function(anchor, heading) {
        var li = document.createElement("li");
        var a = document.createElement("a");
        a.href = "#" + anchor;
        a.textContent = heading.textContent;
        li.appendChild(a);
        heading.innerHTML = "<a name='" + anchor + "'>" + heading.textContent + "</a>";
        heading.className += " no-decoration";
        return li;
    };
    var prevLevel = 0;
    var root, curr;
    headings.forEach(function (heading, index) {
        var tag = heading.tagName.toLowerCase();
        var curLevel = parseInt(tag.replace(/[^\d]/i, ""));// get number from h1, h2, h3,... tags
        var anchor = heading.textContent.replace(/\r?\n|\r/g, "").replace(/\s/, "_"); // remove all new lines and replace spaces with underscore
        var li = createLiWithAnchor(anchor, heading);
        if (curLevel > prevLevel) {
            // open 1 ul and add 1 li
            if (!curr) {
                root = document.createElement("ol");
                root.appendChild(li);
            } else {
                var ul = document.createElement("ul");
                ul.appendChild(li);
                curr.appendChild(ul);
            }
        } else if (curLevel === prevLevel) {
            // add 1 new li next to current li
            curr.parentElement.appendChild(li);
        } else if (curLevel < prevLevel) {
            // close n ul, add one 1 li as a sibling of the ancestor
            var ancestor = closeLevel(curr, prevLevel - curLevel);
            ancestor.parentElement.appendChild(li);
        }
        curr = li;
        prevLevel = curLevel;
    });
    toc.appendChild(root);
}
If you are pasting this into blogger, the function must be XML escaped. This site helped a lot

Css

So it looks a bit better.
/* ----- TOC ----- */
.toc {
background-color: #dadada;
border: gray solid 1px;
margin-bottom: 1em;
}
.toc .toc-title {
margin-left: 1em;
}
.post-body .no-decoration,
.post-body .no-decoration a {
text-decoration: none;
color: black;
}

Usage

Prepare a container

<div id="mytoc"></div>

Call the script

<script type="text/javascript">
TableOfContents("mytoc", "articleId");
</script>
If you using blogger just put the container div at the top of your post and omit the last parameter, the function will assume the article is the parent of the TOC container.

Result

2016/05/17

Gitlab merge request from terminal

This is not a fully automated command for making merge request, instead it is tiny helper command that will take you to the right place in the browser to complete your merge request with one click.


BTW, Why not to do everything with Gitlab API?
It is fully possible. Actually I was trying to do so but in the way I realized that is not what I need because in a merge request you want to set a description (writing a description in the terminal is not better than doing it in the browser), you also will want to assign somebody to review your code (searching for other gitlab users in the terminal is not easier than the browser pulldown menu), you probably want to have a milestone, and maybe other details...

If you still want to do it fully programatically I have some hints at the end of this post :p

Script

#!/usr/bin/env bash -e

# Get host, group and repository names
REMOTE_LINE=`git remote get-url --push origin`
if [[ $REMOTE_LINE == "git@"* ]]; then
    # git: git@my.gitlab.host.com:my-group/project.git
    HOST=$(expr "$REMOTE_LINE" : 'git@\(.*\):.*')
    GROUP=$(expr "$REMOTE_LINE" : 'git@.*:\(.*\)\/.*')
    PROJECT_NAME=$(expr "$REMOTE_LINE" : 'git@.*:.*\/\(.*\)\.git')
elif [[ $REMOTE_LINE == "https://"* ]]; then
    # https: https://my.gitlab.host.com/my-group/project.git
    HOST=$(expr "$REMOTE_LINE" : 'https:\/\/\([^\/]*\)\/.*')
    GROUP=$(expr "$REMOTE_LINE" : 'https:\/\/[^\/]*\/\(.*\)\/.*')
    PROJECT_NAME=$(expr "$REMOTE_LINE" : 'https:\/\/.*\/\(.*\)\.git')
else
    echo Unknown protocol from $REMOTE_LINE
    exit 1
fi

# Get current branch name
BRANCH=$(git rev-parse --abbrev-ref HEAD)

# Create new merge request from current branch to default branch
# (Add `-a /Applications/Firefox.app` to open it with a specific browser)
open https://${HOST}/${GROUP}/${PROJECT_NAME}/merge_requests/new?merge_request[source_branch]=${BRANCH}

Setup

Copy above script as git-mr and place it somewhere in the $PATH and make sure it has executable permissions using chmod
chmod +x ~/.bin/git-mr

Usage

git push origin -u feature/myfeature
git mr
Voilà!
(browsers opens and you are just one click away from completing your merge request)

If you still want to do it programatically with Gitlab API, we need a project id an authentication token to be able to call create merge request API.
POST /projects/:id/merge_requests

Get Authentication Token

We can request the user to enter his/her login and password once and store that in the keychain. Below code is how to retrieve a previously stored password from the keychain
# Get credentials
SECOUT=`security 2>&1 find-internet-password -gs ${HOST}`
LOGIN=`expr "$SECOUT" : '.*"acct"="\([^"]*\)"'`
PASSWORD=`expr "$SECOUT" : '.*password: "\([^"]*\)"'`
echo $LOGIN $PASSWORD
With such information we can call session API and get the token for the upcomming requests
#Initialize a session
SESSIONOUT=`curl -X POST "https://${HOST}/api/v3/session?login=${LOGIN}&password=${PASSWORD}"`
TOKEN=`expr "$SESSIONOUT" : '.*"private_token":"\([^"]*\)"'`
echo $TOKEN

Get Project ID

I have tried doing ${GROUP}/${PROJECT_NAME} and ${GROUP}%2F${PROJECT_NAME} without luck so I got the project by using project search API. Unfortunately gitlab search API is not very powerful yet so we need to search results and get the id of the project that exactly matches our project. Here I am using jq
PROJECT_ID=$(curl --header "PRIVATE-TOKEN: ${TOKEN}" 'https://${HOST}/api/v3/projects/?search=${PROJECT_NAME}' | jq '.[] | select( .path_with_namespace == '\"${GROUP}/${PROJECT_NAME}\"' ) | .id')
Now we are almost ready!

Call merge_request API

# Disabled code: Get tracking branch name
#BRANCH=$(git rev-parse --abbrev-ref --symbolic-full-name @{u})
#BRANCH=${TARGET_BRANCH##origin/}

# Get remote target branch
TARGET_BRANCH='develop'

curl --header "PRIVATE-TOKEN: ${TOKEN}" -X POST "https://${HOST}/api/v3/projects/${PROJECT_ID}/merge_requests" --data "source_branch=${BRANCH}" --data "target_branch=${TARGET_BRANCH}" --data "title=Implementation of my feature"
phew...

2016/05/12

Dictionary to JSON string and JSON string to dictionary

Updated for Swift 4 on 2017/09/06 - Because I always end up Googling NSJSONSerialization and its options...

Dictionary to JSON string

The dictionary is converted to Data which contains an UTF8 encoded string inside.
let dictionary = ["nacho": ["1","2","3"]]
let jsonData = try? JSONSerialization.data(withJSONObject: dictionary, options: [])
let jsonString = String(data: jsonData!, encoding: .utf8)
print(jsonString)
{"nacho":["1","2","3"]}
Also it is possible to use .prettyPrinted which is nice for showing the result string to humans, but in reality I almost never use it.

JSON String to Dictionary

The JSON string should be converted to NSData (using UTF8 encoding), then we can create a dictionary from such data.
let jsonString = "{\"nacho\":[\"1\",\"2\",\"3\"]}"
let jsonData = jsonString.data(using: .utf8)!
let dictionary = try? JSONSerialization.jsonObject(with: jsonData, options: .mutableLeaves)
print(dictionary)
Optional(["nacho": ["1", "2", "3"]])
In JSONReadingOptions there is also .mutableLeaves, .allowFragments, etc but I don't use them very often (Mutable leaves will create a mutable dictionary which might sound helpful but in reality you want to handle things as few as possible in dictionaries since its information is not always statically typed and programmers are able to add/remove key/values without the compiler knowing about it).

Accessing information from objects created with JSONSerialization.jsonObject

Now, since JSONSerialization.jsonObject returns an object of type Any we need to cast it (according to our needs) to be able to access its information. For example:
if let personsDictionary = dictionary as? [String: Any] {
    print(personsDictionary)
    if let numbers = personsDictionary["nacho"] as? [String] {
        print(numbers)
    }
}
["nacho": ["1", "2", "3"]]
["1", "2", "3"]
Hope it helps.