Carrierwave file path stored in associated model

My structure

class Project 
  has_many :documents MyUploader

  def directory
    ...
  end

class Document
  belongs_to :project
  mount_uploader :physical_doc

  def directory
    folder = File.join(self.project.directory, self.class::SUBFOLDER)
    Dir.mkdir(folder) unless File.exists?(folder)
    folder
  end

MyUploader
 def store_dir
    nil
  end

  def cache_dir
    self.model.directory
  end

The storage dir of my document is only known by the project (the location may change, and the Project class is responsible for moving docs in case)

The problem

When deleting a document, Rails will delete the "fields" in the order in which they appear in the code. That is to say

1. It will first delete the :project association
2. Then it will delete the mount_uploader :physical_doc
3. ... But the carrierwave mount_uploader needs to know where the file is stored in order to delete the file ! Oh oh.....

In other words, those two pieces of code are not equivalent

class Document mount_uploader :physical_doc belongs_to :project # This allows the document to be deleted, not created

class Document belongs_to :project mount_uploader :physical_doc # This allows the document to be created, not deleted

Solution ?