Advanced orderby with multiple meta_query

I am on latest version of WordPress. I have scoured the codex concerning wp_query and meta_query. I also read through these:

Query improvements in WP 4.2: ‘orderby’ and ‘meta_query’

A more powerful ORDER BY in WordPress 4.0

Here are my $args for the query:

$args = array (
        'post_type'              => array( 'member' ),
        'tax_query'              => array(
                array(
                    'taxonomy'   => 'group',
                    'field'      => 'slug',
                    'terms'      => array('town-board'),
                    )
                ),
        'meta_query'             => array(
            'relation'           => 'OR',
            'chairman' =>   array(
                    'key'        => '_town-board_position',
                    'value'      => 'Town Chairman',
                    ),
            'supervisor' =>   array(
                    'key'        => '_town-board_position',
                    'value'      => 'Supervisor',
                    'compare'    => 'LIKE',
                    ),
                ),
        'orderby'                => array(
            'supervisor'         => 'ASC',
            'chairman'           => 'DESC',
            ),
    );

I have tinkered with different orderby configurations, but it doesn’t seem to produce the results I would like.

With the $args above I get the chairman listed first but the Supervisors are listed backwards. So, it goes Chairman, Supervisor 3, Supervisor 2, Supervisor 1.

I want to sort this so that chairman is first and then Supervisor 1, Supervisor 2 and so on are listed in order below the chairman.

I just can’t seem to wrap my head around this…

Any input would be appreciated. If more details are needed please let me know.

I tried everything I could to try sorting within the WordPress functions, but was unable to get what I wanted. I ended up following PieterGoosen’s advice and do a usort(). Here is what I have now:

WP_Query $args

$args = array (
        'post_type'              => array( 'member' ),
        'tax_query'              => array(
            array(
                    'taxonomy'   => 'group',
                    'field'      => 'slug',
                    'terms'      => 'town-board'
                ),
        ),
    );

Custom sort function

function custom_sort($a, $b) {
        if ( get_post_meta($a->ID, '_town-board_position', true) === 'Town Chairman' ) {
            return -1;
        } else if ( strpos( get_post_meta($a->ID, '_town-board_position', true), 'Supervisor' ) !== false and strpos( get_post_meta($b->ID, '_town-board_position', true), 'Supervisor' ) !== false ) {
            return strcmp(get_post_meta($a->ID, '_town-board_position', true), get_post_meta($b->ID, '_town-board_position', true));
        } else {
            return 1;
        }
    }

Now, it sorts as I wanted Town Chairman > Supervisor 1 > 2 > 3 > 4

Thank you everyone for your suggestions.