Can I create custom fields for a specific term of a taxonomy which is associated to post or post type?

Yes, you can.

In the `Add Field Group` section choose post/page/post type, then select an associated taxonomy, then select desired term and then create a field group.

Under this group, you can create desired fields.