Tasks and Projects in Task Manager can be deleted as long as the Task Manager User has the appropriate permissions.
Permissions
To have the ability to delete tasks and projects, the Task Manager user must be assigned a Manager role. This can be done by locating the user under Partner Center > Fulfillment > Users or Partner Center > Administration > My Team, then hitting to the right and clicking on Edit User, then Manager.
OR
Deleting Projects Inside Task Manager
Once made a Manager, the Task Manager User can navigate to the project view in Task Manager, and then click on the dropdown to show all of the tasks within a project. Click the (beside "Add Task"), then select Delete Project.
Deleting Tasks Inside Task Manager
Deleting individual tasks can be done through Tasks tab > click on the 3 dots next to the Task > Delete.