Filter medias from the Media Uploader (wp.media) modal with a post meta

I would like to have a media uploader modal that displays only image medias having a specific post meta, let’s say ‘my_image_meta’.
How could I achieve this ?
Here’s my actual code, which just filters images:

    var mediaUploader;
    var $inputVal = $('input[name="geoposts_overlay_ids"]');
    var $inputBt = $('#geoposts_overlays_select');
    $inputBt.click(function(e) {
      e.preventDefault();
        if (mediaUploader) {
        mediaUploader.open();
        return;
      }
      mediaUploader = wp.media.frames.file_frame = wp.media({
        title: overlayUploadL10n.title,
        button: {
          text: overlayUploadL10n.button
        },
        library: {
          type: [ 'image' ]
        },
        multiple: true
      });
      mediaUploader.on('select', function() {
        var selection = mediaUploader.state().get('selection');
        var media_ids = [];

        selection.map( function( getImage ) {
          var media = getImage.toJSON();
          media_ids.push(media.id);
        })

        var media_ids_str = media_ids.join();
        $inputVal.val(media_ids_str);
      });

      mediaUploader.open();
    });

Simplest part is to filter attachment list. Use this code

add_filter('ajax_query_attachments_args', function($args) {
    $args['meta_query'] = array(
        array(
            'key' => 'my_image_meta',
            'value' => $some_value,
            'compare' => '='
        )
    );
    return $args;
});

However this code will filter all media library queries. You may want to pass some additional parameter from wp.media. This answer may be helpful.