QUESTION: In McCardle, the SCOTUS listened to the legislature because of the exceptions clause and agreed to repeal their jurisdiction of issuing writs of habeus corpus. But in Marbury, the SCOTUS decided that the Judiciary Act giving original jurisdiction for writs of mandamus is unconstitutional, implying that the Judiciary Act falls outside the exceptions clause. I don't remember the SCOTUS explicitly reasoning in Marbury why the Judiciary Act falls outside the exceptions clause. What makes something passed by Congress fall inside (or outside) the exceptions clause?
ANSWER: A couple points in response. First, the Court in Marbury did not hold the entire Judiciary Act to be unconstitutional. It merely held that the provision attempting to confer original jurisdiction on the Supreme Court in all cases where mandamus is the appropriate remedy is unconstitutional (or at least unconstitutional as applied in cases like Marbury, where the controversy does not otherwise meet the criteria for original jurisdiction under Art. III, sec. 2, clause 2).
Second, the Exceptions Clause grants Congress the authority to make "exceptions" to, and "regulations" of, the Supreme Court's appellate jurisdiction. A law purporting to give the Court original jurisdiction in a given class of cases simply could not be an exception to, or a regulation of, the Court's appellate jurisdiction.
Finally, to address your actual question, "What makes something passed by Congress fall inside (or outside) the exceptions clause?" I don't think we have a good answer. McCardle provides a very partial response, but it leaves a great deal undecided. Certainly it must plausibly be a regulation of the Court's appellate jurisdiction. But can it completely cut off any path to the Supreme Court for certain constitutional claims? Can it be done selectively, to produce the "underenforcement" of certain constitutional rights, or to undermine the practical force of certain judicial decisions? These questions remain largely unanswered.