Exception originated from within Sacred. KeyError: 'Invalid key "DogmaticDict.__setitem__". Config-keys cannot contain "." or start with "$"'

Hey,

I have some issues with the originated from within Sacred when i debug my code.

ex = sacred.Experiment('Global (train)', ingredients=[data_ingredient, model_ingredient],interactive=True)
SETTINGS.CAPTURE_MODE = 'sys'
ex.captured_out_filter = apply_backspaces_and_linefeeds
@ex.automain
def main(epochs, cpu, cudnn_flag, temp_dir, seed, no_bias_decay):
       '...'

When i debug until the line of @ex.automain, the error was raised as follows.

Exception originated from within Sacred.
Traceback (most recent calls):
File "/root/anaconda3/lib/python3.10/site-packages/sacred/config/utils.py", line 42, in assert_is_valid_key
raise KeyError(
KeyError: 'Invalid key "DogmaticDict.setitem". Config-keys cannot contain "." or start with "$"'

I find the error happen in line 42, in assert_is_valid_key

def assert_is_valid_key(key):
        if SETTINGS.CONFIG.ENFORCE_KEYS_MONGO_COMPATIBLE and (
                isinstance(key, str) and ("." in key or key[0] == "$")
            ):
                raise KeyError(
                    'Invalid key "{}". Config-keys cannot '
                    'contain "." or start with "$"'.format(key)
                )

And the key is defined as:

def normalize_or_die(obj):
    if isinstance(obj, dict):
        res = dict()
        for key, value in obj.items():
            assert_is_valid_key(key)
            res[key] = normalize_or_die(value)
        return res
    elif isinstance(obj, (list, tuple)):
        return list([normalize_or_die(value) for value in obj])
    return normalize_numpy(obj)

But i cannot find the definition of obj. And I cannot find any solution online.

I would be grateful for some help!

Best,

Hey! What does your config look like?

@thequilo

@thequilo
Hey！the same problem
My config is as follow:

def _loss_names(d):
    ret = {
        "itm": 0,
        "mlm": 0,
        "mpp": 0,
        "vqa": 0,
        "nlvr2": 0,
        "irtr": 0,
    }
    ret.update(d)
    return ret
    
@ex.config
def config():
    exp_name = "vilt"
    seed = 0
    datasets = ["coco", "vg", "sbu", "gcc"]
    loss_names = _loss_names({"itm": 1, "mlm": 1})
    batch_size = 4096  # this is a desired batch size; pl trainer will accumulate gradients when per step batch is smaller.

    # Image setting
    train_transform_keys = ["pixelbert"]
    val_transform_keys = ["pixelbert"]
    image_size = 384
    max_image_len = -1
    patch_size = 32
    draw_false_image = 1
    image_only = False

    # Text Setting
    vqav2_label_size = 3129
    max_text_len = 40
    tokenizer = "bert-base-uncased"
    vocab_size = 30522
    whole_word_masking = False
    mlm_prob = 0.15
    draw_false_text = 0

    # Transformer Setting
    vit = "vit_base_patch32_384"
    hidden_size = 768
    num_heads = 12
    num_layers = 12
    mlp_ratio = 4
    drop_rate = 0.1

    # Optimizer Setting
    optim_type = "adamw"
    learning_rate = 1e-4
    weight_decay = 0.01
    decay_power = 1
    max_epoch = 100
    max_steps = 25000
    warmup_steps = 2500
    end_lr = 0
    lr_mult = 1  # multiply lr for downstream heads

    # Downstream Setting
    get_recall_metric = False

    # PL Trainer Setting
    resume_from = None
    fast_dev_run = False
    val_check_interval = 1.0
    test_only = False

    # below params varies with the environment
    data_root = ""
    log_dir = "result"
    per_gpu_batchsize = 0  # you should define this manually with per_gpu_batch_size=#
    num_gpus = 1
    num_nodes = 1
    load_path = ""
    num_workers = 8
    precision = 16
    
    
@ex.named_config
def task_finetune_vqa_randaug():
    exp_name = "finetune_vqa_randaug"
    datasets = ["vqa"]
    train_transform_keys = ["pixelbert_randaug"]
    loss_names = _loss_names({"vqa": 1})
    batch_size = 256
    max_epoch = 10
    max_steps = None
    warmup_steps = 0.1
    draw_false_image = 0
    learning_rate = 1e-4
    val_check_interval = 0.1
    lr_mult = 10

@Wznnnnn

Hey @Wznnnnn! I just copied your config and added a simple main function. It works for me, I don't see the error. Can you provide a smaller example that reproduces the error?

It could also be related to the way you call the script. Another reason could be a debugger, which sometimes injects symbols into the local namespace. Weird things can happen if you add a breakpoint in a config scope.

Navigation Menu

Repository navigation

Exception originated from within Sacred. KeyError: 'Invalid key "DogmaticDict.setitem". Config-keys cannot contain "." or start with "$"' #920

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Exception originated from within Sacred. KeyError: 'Invalid key "DogmaticDict.__setitem__". Config-keys cannot contain "." or start with "$"' #920

Description

Activity

thequilo commented on Aug 29, 2023

Wznnnnn commented on Jun 8, 2024

thequilo commented on Jun 10, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Exception originated from within Sacred. KeyError: 'Invalid key "DogmaticDict.setitem". Config-keys cannot contain "." or start with "$"' #920