Can a table have a foreign key without a primary key?

Does a table with a foreign key need a primary key?

A foreign key is a set of one or more columns in any table (not necessarily a candidate key, let alone the primary key, of that table) which may hold the value(s) found in the primary key column(s) of some other table. So we must have a primary key to match the foreign key.

Can you have a table with just a foreign key?

There is no problem having a table that consists of foreign keys only. In your particular example it would be OK to remove the OrderID. Relational theory states that each table should have a candidate key so, ideally, the business rule you mention would be in place to make the foreign key only table a "good" table.