datagrok-df-and-columns

name: datagrok-df-and-columns description: Find, describe, add, remove, rename, clone, or set metadata on columns of a Datagrok DataFrame inside a datagrok-exec block. Use whenever the user asks to locate "the X column", summarize a column, add a typed/empty/values-filled/virtual column, set semantic type / units / format / friendly name, apply linear or categorical or conditional color coding, drop or rename columns, or copy a DataFrame. Covers everything in DataFrame.columns and Column.meta — but not row filtering/selection (datagrok-filtering, datagrok-selection) and not formula-only columns (datagrok-calc-column).

datagrok-df-and-columns

Use the DG.DataFrame / DG.Column js-api inside a datagrok-exec block. Globals available in the block: grok, ui, DG, view, t (the current DG.DataFrame, when view.type === 'TableView').

For formula-driven columns (recompute on source change), use datagrok-calc-column. Row filtering / selection lives in datagrok-filtering and datagrok-selection.

Quick reference

Finding the right column

t.col and t.columns.byName are case-insensitive. Always pass the semType constant (DG.SEMTYPE.MOLECULE), never the string literal 'Molecule'.

// Find the molecule column. bySemType returns the first match or null.
const molCol = t.columns.bySemType(DG.SEMTYPE.MOLECULE);
return ui.divText(molCol ? `Molecule column: ${molCol.name}` : 'No molecule column');

// User said "mw" but the DF has "Molecular Weight" — try exact first, then
// fall back to a name substring match.
const wanted = 'mw';
let col = t.col(wanted);
if (!col) {
  const lc = wanted.toLowerCase();
  col = t.columns.firstWhere((c) => c.name.toLowerCase().includes(lc));
}
return ui.divText(col ? col.name : 'no match');

Describing a column

col.stats is cached per column — repeated access is cheap.

Do not compute these by hand:

• Total rows: col.length, not col.toList().length (allocates).
• Non-null count: col.stats.valueCount.
• Distinct count: col.stats.uniqueCount, not new Set(col.toList()).size.
• For string columns, col.categories.length === col.stats.uniqueCount.

// Build a key-value summary of one numeric column.
const col = t.getCol('age');
const s = col.stats;
return ui.tableFromMap({
  name: col.name,
  type: col.type,
  semType: col.semType ?? '(none)',
  length: col.length,
  missing: s.missingValueCount,
  unique: s.uniqueCount,
  min: s.min, max: s.max, avg: s.avg, stdev: s.stdev,
});

// Summarize every numeric column. t.columns.numerical is Iterable<Column>
// (not an Array) — convert via Array.from to get .map etc.
const rows = Array.from(t.columns.numerical, (c) => ({
  name: c.name, min: c.stats.min, max: c.stats.max, avg: c.stats.avg,
}));
return DG.Viewer.grid(DG.DataFrame.fromObjects(rows)).root;

Top-N categories

No built-in top-N for a string column. Tally into a Map<string, number>:

const col = t.getCol('class');
const counts = new Map();
for (let i = 0; i < col.length; i++) {
  const v = col.get(i);
  if (v === null || v === undefined) continue;
  counts.set(v, (counts.get(v) ?? 0) + 1);
}
const top = Array.from(counts.entries())
  .sort((a, b) => b[1] - a[1])
  .slice(0, 5);
return ui.tableFromMap(Object.fromEntries(top));

If you only need distinct values (no counts), col.categories is already sorted alphabetically.

Cloning DataFrames

t.copy() does not exist. Use t.clone(rowMask?, columnIds?, saveSelection?, saveTags?):

// Just SMILES and activity, filtered rows only.
const small = t.clone(t.filter, ['smiles', 'activity']);
return DG.Viewer.grid(small).root;

col.clone(mask?) clones a single column, but the cloned column is not attached to any DataFrame. Call df.columns.add(clonedCol) to attach it.

Adding columns

DG.COLUMN_TYPE.FLOAT === 'double'. Never write the literal 'float' (no such type) or 'double' (works but brittle). Use the constant, or the typed shorthand addNewFloat.

// Typed empty column with metadata applied in the same block.
const col = t.columns.addNewFloat('Ki');
col.semType = DG.SEMTYPE.Ki;
col.meta.units = 'nM';
col.meta.format = '0.00';
return ui.divText(`Added ${col.name} (${col.type})`);

// Add a column from an array of values. fromList infers length from the array.
const values = [1.2, 3.4, null, 5.6];
const col = DG.Column.fromList(DG.COLUMN_TYPE.FLOAT, 'score', values);
t.columns.add(col);
return ui.divText(`Added ${col.name}`);

// Virtual column — recomputed on every access, not stored. Type argument is
// DG.TYPE.* (not DG.COLUMN_TYPE.*) — addNewVirtual takes the broader enum.
const labels = t.columns.addNewVirtual('Label',
  (i) => `${t.get('compound', i)}@${t.get('target', i)}`,
  DG.TYPE.STRING);
return ui.divText(`Added virtual column ${labels.name}`);

// Avoid name collisions explicitly.
const name = t.columns.getUnusedName('Ki');  // returns 'Ki' or 'Ki (1)' etc.
const col = t.columns.addNewFloat(name);
return ui.divText(`Final name: ${col.name}`);

Removing & renaming

// Remove one column. Accepts string, index, or Column instance.
t.columns.remove('temp');

// Remove several. Loop — no bulk remove call exists.
for (const name of ['_tmp1', '_tmp2', 'temp3'])
  if (t.columns.contains(name))
    t.columns.remove(name);

To target calculated columns, check col.meta.formula:

// Drop every calculated column.
const names = t.columns.toList().filter((c) => c.meta.formula != null).map((c) => c.name);
for (const n of names) t.columns.remove(n);
return ui.divText(`Removed: ${names.join(', ') || '(none)'}`);

Rename: assign col.name directly.

t.getCol('pIC50').name = 'pIC50_old';

Setting column metadata

Use the right accessor — never setTag for these.

Assign null to a col.meta.* accessor to clear the underlying tag.

Trap. The semType tag key is 'quality' in storage. Don't use col.setTag('semType', ...) or col.setTag('quality', ...). Use the col.semType accessor — it sets the right tag and fires the right event. Friendly name / format / units / description go through col.meta.*, not through setTag.

// Set several metadata fields at once.
const col = t.getCol('Ki');
col.semType = DG.SEMTYPE.Ki;
col.meta.units = 'nM';
col.meta.format = '0.00';
col.meta.description = null;  // null clears

// Clear all custom (user-facing) metadata. Do NOT call col.tags.clear() —
// that also nukes system tags like .color-coding-type.
const col = t.getCol('activity');
col.meta.friendlyName = null;
col.meta.units = null;
col.meta.format = null;
col.meta.description = null;

Color coding

Lives under col.meta.colors. Four shapes:

Linear (numeric only — throws on string)

// Red → yellow → green across the value range, with min/max pinned.
t.getCol('age').meta.colors.setLinear(
  ['#ff0000', '#ffff00', '#00ff00'],
  {min: 19, max: 70},
);

Hex strings and ARGB integers both work in range. Optional belowMinColor / aboveMaxColor paint out-of-range values.

Categorical

t.getCol('race').meta.colors.setCategorical(
  {'Asian': '#0000FF', 'Black': '#FF0000'},
  {fallbackColor: '#CCCCCC'},
);

Conditional (range-based rules)

// '20-170' means "value in the range 20..170". '<100' / '>50' also work.
t.getCol('height').meta.colors.setConditional({
  '20-170': '#00FF00',
  '170-190': '#220505',
});

Off

t.getCol('activity').meta.colors.setDisabled();

Trap. Don't use col.setTag('.color-coding-type', ...) to enable / disable coloring. It bypasses rule validation and the companion tag writes that setLinear / setCategorical / setConditional perform, producing broken color coding.

Working with cell values

Bulk init: col.init(scalar | (i) => value) is the right shape for an already-allocated column.

const col = t.columns.addNewInt('rowSquared');
col.init((i) => i * i);

For hot paths, mutate values with notify=false and call col.fireValuesChanged() once at the end:

const src = t.getCol('input');
const dst = t.getCol('output');
for (let i = 0; i < t.rowCount; i++)
  dst.set(i, src.get(i) * 2, false);
dst.fireValuesChanged();

Constants — cite, never invent

Anti-patterns — and the fix

1. Looping t.rows to read / write column data. Slow and notification-heavy. Use t.getCol(name) and an index loop with set(..., false) plus a final col.fireValuesChanged().
2. col.setTag('color-coding-type', ...) / col.setTag('.color-coding-type', ...). Use col.meta.colors.setLinear/setCategorical/setConditional/setDisabled.
3. col.setTag('semType', 'Molecule'). Use col.semType = DG.SEMTYPE.MOLECULE.
4. col.setTag('friendlyName', '...'), col.setTag('format', '...'). Go through col.meta.friendlyName / col.meta.format.
5. 'float' or 'double' string literals for column type. Always DG.COLUMN_TYPE.FLOAT (or just addNewFloat).
6. t.copy(). Doesn't exist. Use t.clone().
7. col.toList().length for row count. Use col.length, or col.stats.valueCount for non-null count.
8. new Set(col.toList()).size for unique count. Use col.stats.uniqueCount. For string cols, col.categories.length also works.
9. Mutating col.getRawData() and forgetting to fire. End with col.fireValuesChanged() — otherwise viewers / grid don't refresh.
10. Forgetting await on t.columns.addNewCalculated. It's the only addNew* that returns a Promise<Column>. See datagrok-calc-column.
11. t.columns.byName(name) + if (col == null) when you want an error. Use t.getCol(name) — it throws on missing.
12. col.tags.clear() to "reset metadata". Nukes system tags too (color coding, choices, etc). Clear specific col.meta.* fields with null.