Next/Previous Links in same category

I’ve been pouring over this problem that seems really minor, but I can’t seem to solve it. I’m trying to make next/previous links that only link to the category that the post is in. This is for a custom post type called portfolio. This is the code I currently have on the content-portfolio page:

<?php next_post_link( '<div class="nav-previous">%link</div>', '<span class="meta-nav">' . _x( '←', 'Next post link', 'morphology' ) . '</span> %title'); ?>
<?php previous_post_link( '<div class="nav-next">%link</div>', '%title <span class="meta-nav">' . _x( '→', 'Previous post link', 'morphology' ) . '</span>'); ?>

I’ve seen some variations of the following, but I can’t seem to edit it to get it to cooperate:

<?php next_post_link( '%link', 'Next post in category', TRUE ); ?>

Any advice as to how to make this work or what I’m doing wrong?

Update:
I still haven’t gotten a good solution to this, and nothing I’ve tried from the other top Google hits seem to be working. As soon as I say that in_same_term = true, the link disappears entirely. Based on the answer below, this is the code I’m using:

<?php next_post_link( 
'<div class="nav-previous">%link</div>', 
'<span class="meta-nav">' . _x( '←', 'Next post link', 'morphology' ) . '</span> %title', true, array(), 'portfolio'); ?>

previous_post_link takes 5 params, but you use only 2 of them.

Let’s take a look at other 3:

• in_same_term (boolean) (optional) Indicates whether previous post must
  be within the same taxonomy term as the current post. If set to
  ‘true’, only posts from the current taxonomy term will be displayed.
  If the post is in both the parent and subcategory, or more than one
  term, the previous post link will lead to the previous post in any of
  those terms. true false Default: false
• excluded_terms (string/array)
  (optional) Array or a comma-separated list of numeric terms IDs from
  which the next post should not be listed. For example array(1, 5) or
  ‘1,5’. This argument used to accept a list of IDs separated by ‘and’,
  this was deprecated in WordPress 3.3 Default: None
• taxonomy (string)
  (Optional) Taxonomy, if $in_same_term is true. Added in WordPress 3.8.
  Default: ‘category’

So if you want to navigate through posts from the same category, third param should be set to true.

<?php
    previous_post_link(
        '<div class="nav-next">%link</div>',
        '%title <span class="meta-nav">' . _x( '→', 'Previous post link', 'morphology' ) . '</span>',
         true
    );
?>

And if your portfolio uses custom taxonomy, you should point that out in last param:

<?php
    previous_post_link(
        '<div class="nav-next">%link</div>',
        '%title <span class="meta-nav">' . _x( '→', 'Previous post link', 'morphology' ) . '</span>',
         true,
         array(),
         'my_custom_taxonomy'
    );
?>

It works the same with next_post_link.