Tasks and Projects in Task Manager can be deleted as long as the Task Manager User has the appropriate permissions.
Watch the Video Walkthrough
Permissions
To have the ability to delete tasks and projects, the Task Manager user must be assigned a Manager role. This can be done by locating the user under Partner Center > Tasks > Users or Partner Center > Administration > My Team, then hitting 
to the right and clicking on "Edit User", then "Manager".
OR
Deleting Projects Inside Task Manager
Once made a Manager, the Task Manager User can navigate to the project view in Task Manager, then click on the dropdown to show all of the tasks within a project. Click the (beside "Add Task"), then select Delete Project.
Deleting Tasks Inside Task Manager
Deleting individual tasks can be done through Accounts > click on the account name > View all tasks. In this view, the Task Manager user can click the checkboxes of the tasks they would like to delete, then a "Delete" button will appear across from the search bar.
Video Walkthrough