XML Project¶
The core technical documentation for sHEL's literal-safe data handling system. This project addresses the fundamental problem of storing and transmitting data that contains cmd.exe special characters without triggering unwanted interpretation by the command-line parser.
The original project¶
set "var=["'I am a hater of XML'"]" && echo %var%| clip
<# ["'I am a hater of XML </s>'"] <--- added to clipboard.
What is the use case?¶
XML can store a large sum of data. This project was initially justified due to being able to consecutively use basic dos shell, clip, and markdown, to store an insane amount of data as an alternative to SQL databasing. - Why not store a dumb amount of stuff into variables and keep things only in temporary memory, where it's assumed clipboard history is disabled? clip is powerful. So is echo.
Here is an example of databasing:¶
set "x=[""]"
set "y=['']"
set "z=["''"]"
set "0aaa=<111somehtml line 1>"
set "0aab=<222somehtml this is line 2>"
set "0aac=<333somehtml with </a> extras>"
set '0aad=%x:~0,2%%0aaa%%x:~2%'&set '0aae=%x:~0,2%%0aab%%x:~2%'&set '0aaf=%x:~0,2%%0aac%%x:~2%'&set "0aba=%y:~0,2%%0aaa%%y:~2%"&set "0abb=%y:~0,2%%0aab%%y:~2%"&set "0abc=%y:~0,2%%0aac%%y:~2%"&set 0abd=%z:~0,3%%0aaa%%z:~3%&set 0abe=%z:~0,3%%0aab%%z:~3%&set 0abf=%z:~0,3%%0aac%%z:~3%
<# This is the same as:
set '0aad=%x:~0,2%%0aaa%%x:~2%' <-- Sets ["<html>"] for string searching
set '0aae=%x:~0,2%%0aab%%x:~2%'
set '0aaf=%x:~0,2%%0aac%%x:~2%'
set "0aba=%y:~0,2%%0aaa%%y:~2%" <-- Sets ['<html>'] for delimiting
set "0abb=%y:~0,2%%0aab%%y:~2%"
set "0abc=%y:~0,2%%0aac%%y:~2%"
set 0abd=%z:~0,3%%0aaa%%z:~3% <-- Sets ["'<html>'"] for storage --> (see %0abd:~3,-3% for extracting previous data, and compatibility with all but delimitation)
set 0abe=%z:~0,3%%0aab%%z:~3%
set 0abf=%z:~0,3%%0aac%%z:~3%
#>
Tokenize, Search, and Run Logic:¶
set "search=for /f "tokens= delims=[" %A in ("") do @for /f "tokens=1 delims=]" %B in ("%A") do @echo %B"
set tok=%search:~0,15%
set aa=%search:~15,19%
set bb=%search:~34%
%search:~0,15% [Enter Tokens to Dig] %search:~15,19% [Paste Large Block to Delimit nth value] %search:~34%
# OR:
%tok% [Enter Tokens to Dig] %aa% [Paste Large Block to Delimit nth value] %bb%
Example
Searching for second list item (tok=3 == item 2):
%tok%3%aa%(echo ['<jibberish 1>']&echo ['<wtf ! /2>']&echo ['69 86 <64????/> <third value>']&echo ['249 86 64 ^*!?<this is the fourth line, etc.></a>'])| clip%bb%
['<wtf ! /2>']
Note
- +1 needs to be added to the search. 0 returns nothing. 1 returns everything prior to the list. tok=2 returns first. tok=3 returns second, etc.
- Secondly, x and z format work for simple lists but have limitations. y formatting [''] works all the time, but lacks searchable strings.
Searchable strings
(echo ["'<somehtml>'"]&echo ["'"%date%,%time:~0,8%"... is actually the time <testhtml>'"]&echo ["'<somehtml> #3'"]) | findstr "3"
["'<somehtml> #3'"]
(echo ["'<somehtml>'"]&echo ["'"%date%,%time:~0,8%"... is actually the time <testhtml>'"]&echo ["'<somehtml> #3'"]) | findstr "time"
["'"%date%,%time:~0,8%"... is actually the time <testhtml>'"]
Note
This only works for long echo strings, and usually fails if searching a piped or mega-one-liner clip command.
Boolean cmd — Click to Expand
Now that we have the basics, why not add if/then functionality to dos shell by being clever?
Removing/I allows for a case-sensitive variant.
Now, run all the macros you like!
We can paste a large multi-line list with clip,
search a string out of it,
if it exists, pipe to clip something else.
Possibility when running macros? Run a dummy "about:blank" paste instead of a URL if something is not found, then, add to clipboard large, multiline table that is markdown friendly (containing html elements) — your database.
echo "text to search" | find /I "needle" >nul && (echo Great Success! & (echo ["| <a href="https://example.com" title="this is sparta, no, this is a generic T icon"><img src="data:image/png; base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACpUlEQVR4nGxTz0tVQRg937z79Hnx0SLNcvEMXBRFq4x+UEKE9EsXbWoR/XLxKqhFEfUHtMpthSiBYUG0kqiNKBFqmBBBWtSiLF9l+CsR7Jm+e+frzL2KL2gud+bemXPOfN+Z +TywKSASDewuYh+Hc/yp52RGA/ 4tIMfvPqxBh9xGfzFHsNy0Cb7NoMWIZJE0SVgDFBRakYFsPQhMjADvBwrw0I4crstT5B1PnNLEKfjr0uiSMjRw2uoiLEIkUODi5fuwu8643QK5e8jgY7dRJHom58JjVQ+Q91wYYTluiU/yHJaQTnvY0ehJ9TZVy/ j5mqFO1Z2nPWw5bDHUvSTrpaEyJAe4JHoeu5GUPoYrqN4k9uqAkfKKyBCJc41GCRZVWxtFhnstUi5wVUZY71lB1njMrIAAGhjz9ZXq4EORzfuBPc2QzrOA58P+GhXz7rmilHpWLZLwGFyWntH1gGoejP78LHqnSbT3MZCfgVJZh7uA35OQk/ egR66J5imQSBh6pI7rBDK0TKKXeClJQvwEd03FodfyVC88cU4BP0YgxMDaGE+ut5Kj6+KcNQYzyuikZ74AH3qhn17AvOwGKikehMskwBCfi/pVympjZkhXMSwDGXsDLTdMn2Y7vIn2yhli +nnijmeNm1LVmL0stXYjpCwNzH6DGKvGrTMJx3FcQ1I7DQmijFB8ehon9JYmTjONuhPKK+0M1AhLjuMaacOgDdGGFDXVCYmFg2jI3sAcvanYfhyYnyKZOyurg1jHcVzjrvLUPG7oH/ TARwkoLSkKvX5E0rTqgSuis98VI88cESgNSxw24hR7Np6FvyGJFn6ymPgsMIGqWqCmDjLKApwdd5YXLMOeYDFVrxTT/8qZoGaTkL283jWuoCg3xgrt5/l3SOu/5fwXAAD//5rxYZkAAAAGSURBVAMAkjI5uqP0pJcAAAAASUVORK5CYII=" alt="An image on the screen displays a generic 'T', This part is read by accessibilty screen readers" style="width:24px;height:24px;"></a><br><span style="font-size:13px; color:#666;">Figure 1 — Example icon footer</span> | Test | Three |"]&echo ["|---|---|---|"]&echo ["| aaa | bbb | ccc |"]&echo ["| ddd | eee | fff |"]&echo ["#### Example title"])| clip) || (set "var=This element does not exist" & echo %var% & echo %var%| clip)
[" and "] due to html elements—find and replace these easily).
Figure 1 - Example icon footer |
Test | Three |
|---|---|---|
| aaa | bbb | ccc |
| ddd | eee | fff |
Example title¶
blank line
If there is no needle. The command prints and clips custom text. Note, echo off | clip will clear the clipboard.
Here is a plaintext rendering of the output after removing delimiters:
A reproducable structure (step-by-step):¶
Set your database:
set "x=[""]"
set "y=['']"
set "z=["''"]"
set "0aaa=Title: Task 1 DB: My Datasheet"
set "0aab=| Index | TodoCount | ThinkCount | TermCount | PyCount | BufferVar |"
set "0aac=|-|-|-|-|-|-|"
set "0aad=| 1aaa # | 0aaa # | 0aab # | 0aac # | 0aad # | 0aae # |"
set "x=[""]"&set "y=['']"&set "z=["''"]"&set "0aaa=Title: Task 1 DB: My Datasheet"&set "0aab=| Index | TodoCount | ThinkCount | TermCount | PyCount | BufferVar |"&set "0aac=|-|-|-|-|-|-|"&set "0aad=| 1aaa # | 0aaa # | 0aab # | 0aac # | 0aad # | 0aae # |"
set 0aba=%x:~0,2%%0aaa%%x:~2%&set 0abb=%x:~0,2%%0aab%%x:~2%&set 0abc=%x:~0,2%%0aac%%x:~2%&set 0abd=%x:~0,2%%0aad%%x:~2%&set "0aca=%y:~0,2%%0aaa%%y:~2%"&set "0acb=%y:~0,2%%0aab%%y:~2%"&set "0acc=%y:~0,2%%0aac%%y:~2%"&set "0acd=%y:~0,2%%0aad%%y:~2%"&set 0ada=%z:~0,3%%0aaa%%z:~3%&set 0adb=%z:~0,3%%0aab%%z:~3%&set 0adc=%z:~0,3%%0aac%%z:~3%&set 0add=%z:~0,3%%0aad%%z:~3%
set "0baa=Title: Task 1 DB: My Second Datasheet"
set "0bab=| Index | TodoCount | ThinkCount | TermCount | PyCount | BufferVar |"
set "0bac=|-|-|-|-|-|-|"
set "0bad=| 1aaa # | 0aaa # | 0aab # | 0aac # | 0aad # | 0aae # |"
set "0baa=Title: Task 1 DB: My Second Datasheet"&set "0bab=| Index | TodoCount | ThinkCount | TermCount | PyCount | BufferVar |"&set "0bac=|-|-|-|-|-|-|"&set "0bad=| 1aaa # | 0aaa # | 0aab # | 0aac # | 0aad # | 0aae # |"
set '0bba=%x:~0,2%%0baa%%x:~2%'&set '0bbb=%x:~0,2%%0bab%%x:~2%'&set '0bbc=%x:~0,2%%0bac%%x:~2%'&set '0bbd=%x:~0,2%%0bad%%x:~2%'&set "0bca=%y:~0,2%%0baa%%y:~2%"&set "0bcb=%y:~0,2%%0bab%%y:~2%"&set "0bcc=%y:~0,2%%0bac%%y:~2%"&set "0bcd=%y:~0,2%%0bad%%y:~2%"&set 0bda=%z:~0,3%%0baa%%z:~3%&set 0bdb=%z:~0,3%%0bab%%z:~3%&set 0bdc=%z:~0,3%%0bac%%z:~3%&set 0bdd=%z:~0,3%%0bad%%z:~3%
Expand to show — “What you just did”
set 0bba=%x:~0,2%%0baa%%x:~2% # <-- Sets ["<html>"] for string searching
set 0bbb=%x:~0,2%%0bab%%x:~2%
set 0bbc=%x:~0,2%%0bac%%x:~2%
set 0bbd=%x:~0,2%%0bad%%x:~2%
set "0bca=%y:~0,2%%0baa%%y:~2%" # <-- Sets ['<html>'] for delimiting
set "0bcb=%y:~0,2%%0bab%%y:~2%" # <-- c can always be piped to clip
set "0bcc=%y:~0,2%%0bac%%y:~2%"
set "0bcd=%y:~0,2%%0bad%%y:~2%" # <-- c can be nested echo " " --> See: # echo "(echo %0aca%&echo %0acb%&echo%0acc%&echo%0acd%)"
set 0bda=%z:~0,3%%0baa%%z:~3% # <-- Sets ["'<html>'"] for storage --> %0abd:~3,-3% for extracting previous data, and compatibility with all but delimitation)
set 0bdb=%z:~0,3%%0bab%%z:~3%
set 0bdc=%z:~0,3%%0bac%%z:~3% # <-- d can be nested echo "" "" (See: echo ""(echo %0bba%&echo %0bbb%&echo%0bbc%&echo%0bbd%)""
set 0bdd=%z:~0,3%%0bad%%z:~3% # <-- d by itself can be piped to findstr (See: (echo %0bda%&echo %0bdb%&echo%0bdc%&echo%0bdd%) | findstr "index"
...
(echo %0bba%&echo %0bbb%&echo%0bbc%&echo%0bbd%)| clip
(echo %0bca%&echo %0bcb%&echo%0bcc%&echo%0bcd%)| clip
(echo %0bda%&echo %0bdb%&echo%0bdc%&echo%0bdd%)| clip
set 0aba=%x:~0,2%%0aaa%%x:~2% # <-- Sets ["<html>"] for string searching
set 0abb=%x:~0,2%%0aab%%x:~2%
set 0abc=%x:~0,2%%0aac%%x:~2% # <-- b can be nested echo "" "" (See: echo ""(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)"" # <-- will expand (4 lines will be echoed), encapsulated in double quote ""<start ---- end>""
set 0abd=%x:~0,2%%0aad%%x:~2%
set "0aca=%y:~0,2%%0aaa%%y:~2%" # <-- Sets ['<html>'] for delimiting
set "0acb=%y:~0,2%%0aab%%y:~2%" # <-- c can be delimited (walked), but sometimes has string issues.
set "0acc=%y:~0,2%%0aac%%y:~2%" # <-- c can be nested echo " " --> See: # echo "(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%)" # <-- will be echoed as a single oneliner block. Compresses html to single line string.
set "0acd=%y:~0,2%%0aad%%y:~2%"
set 0ada=%z:~0,3%%0aaa%%z:~3% # <-- Sets ["'<html>'"] for storage --> %0abd:~3,-3% for extracting previous data, and compatibility with all but delimitation)
set 0adb=%z:~0,3%%0aab%%z:~3% # <-- d by itself is almost always safe, but b is better.. (See: (echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%) | findstr "index"
set 0adc=%z:~0,3%%0aac%%z:~3% # <-- d can be nested echo "" "" (See: echo ""(echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%)"" # <-- will expand (4 lines will be echoed), encapsulated in double quote ""<start ---- end>""
set 0add=%z:~0,3%%0aad%%z:~3%
...
(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%) # <--almost always perfect, cant delimit
(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%) # <--almost always perfect, can't echo
(echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%) # <--almost always perfect, can't delimit
Create delimit function:
set "search=for /f "tokens= delims=[" %A in ("") do @for /f "tokens=1 delims=]" %B in ("%A") do @echo [%B] && @set "code=%B" && @set "cbuf=[%B]" && @set dbuf=["%B"] && @echo ["%B"]| clip"
set "grabb=echo ["%p%code:~1,-1%p%"]| clip && @echo ["%p%code:~1,-1%p%"]| for /f "delims=" %A in ('more') do @set "bbuf=%A""
Example
This works with c
b and d cannot delimit due to strings
Expand to show — “Why do this?”
# Default: Just echo 'result':
set "search=for /f "tokens= delims=[" %A in ("") do @for /f "tokens=1 delims=]" %B in ("%A") do @echo %B"
# v2.0: Just perform echo of compat(d), store diggable(c), as well as store/clip compat(d) :
set "search=for /f "tokens= delims=[" %A in ("") do @for /f "tokens=1 delims=]" %B in ("%A") do @echo [%B] && @set "code=%B" && @set "cbuf=[%B]" && @set dbuf=["%B"] && @echo ["%B"]| clip"
# Bonus :
set "p=%"
set "grabb=echo ["%p%code:~1,-1%p%"]| clip && @echo ["%p%code:~1,-1%p%"]| for /f "delims=" %A in ('more') do @set "bbuf=%A""
<#
Now running %grabb% adds searchable(b) to library.
Now we have:
bbuf = ["result"]
cbuf = ['result']
dbuf = ["'result'"]
clipboard = ["result"] --> or ["'result'"] if %bbuf% is never run to grab (b).
#>
# "Search" :
for /f "tokens= delims=[" %A in ("") do @for /f "tokens=1 delims=]" %B in ("%A") do @echo [%B] && @set "code=%B" && @set "cbuf=[%B]" && @set dbuf=["%B"] && @echo ["%B"]| clip
# "B Extraction" :
echo ["%p%code:~1,-1%p%"]| clip && @echo ["%p%code:~1,-1%p%"]| for /f "delims=" %A in ('more') do @set "bbuf=%A"
%tok% [Enter Tokens to Dig] %aa% [Paste Large Block to Delimit nth value] %bb%
#Generic Example: <-- Prints and clips delimitation (in this case, second value) <-- bbuf grabs and saves extra buffer zone
#For cmdlet oneliner : (echo ['<jibberish 1>']&echo ['<wtf ! /2>']&echo ['69 86 <64????/> <third value>']&echo ['249 86 64 ^*!?<this is the fourth line, etc.></a>'])| clip
#Run :
%tok%3%aa%_______%bb%
%bbuf%
#Like:
%tok%3%aa%(echo ['<jibberish 1>']&echo ['<wtf ! /2>']&echo ['69 86 <64????/> <third value>']&echo ['249 86 64 ^*!?<this is the fourth line, etc.></a>'])| clip%bb%
%grabb%
<#
Prints (tok=3 : 2nd delimitation):
['<wtf ! /2>']
Stores:
["<wtf ! /2>"] <-- %bbuf%
['<wtf ! /2>'] <-- %cbuf%
["'<wtf ! /2>'"] <-- %dbuf%
["<wtf ! /2>"] <-- clipboard (b) after %grabb% , else clipboard is temporarily %dbuf%
Working Example: <-- Delimits 4th line of html (**normally un-echoable!**)#>
#A:
%tok%5%aa%(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%)| clip%bb%
#B:
%tok%5%aa%(echo %0bca%&echo %0bcb%&echo %0bcc%&echo %0bcd%)| clip%bb%
Create search function:
It's not worthwhile to save, since it's not a hard commandExample
# From the first database:
(echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%) | findstr /I "aaa"
(echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%) | findstr "aaa"
# From the second database
(echo %0bda%&echo %0bdb%&echo %0bdc%&echo %0bdd%) | findstr /I "aaa"
(echo %0bda%&echo %0bdb%&echo %0bdc%&echo %0bdd%) | findstr "aaa"
This works with (b/d)
c has serious issues
Expand to show — “Why do this?”
# How to Search (b/d):
( [ Echo to Search ] ) | findstr /I "text" <-- /I is case insensitive :
() | findstr /I ""
() | findstr ""
# Generic Example:
(echo ["'<somehtml>'"]&echo ["'"%date%,%time:~0,8%"... is actually the time <testhtml>'"]&echo ["'<somehtml> #3'"]) | findstr "3"
# Generic Example 2 -> pipes to clip:
((echo ["'<somehtml>'"]&echo ["'"%date%,%time:~0,8%"... is actually the time <testhtml>'"]&echo ["'<somehtml> #3'"]) | findstr "3")| clip
Create booling function:
set "bool=echo "" | find /I "needle" >nul && (echo Great Success! & ()| clip) || (set "buffer=about:blank" & echo This element does not exist! & echo %buffer%| clip)"
Expand to show — “What you just did”
# Debug is shown on the right. (Echo partial commands safely)
# i.e. echo "%aif%" for echoing the command { <-- " "} vs { <-- "" ""}
set aif=%bool:~0,6% <-- " "
(6-19th)
set ahas=%bool:~6,13% <-- "" "" <-- i.e: echo ""%ahas%""
set "ahas0=%bool:~7,12%" <-- " "
(25-58th)
set athen=%bool:~25,33% <-- "" ""
(60-77th)
# Else: choose buffer..
set "cc=%bool:~60,17%" <-- " " (inside) # always clipped
set "cc0=%bool:~66,11%" <-- " " (inside)| clip # use for custom pipe (success)
set "cc1=%bool:~60,16%" <-- " " # (used for non-string buffers)
set "cc2=%bool:~66,10%" <-- " " # (used for non-string buffers)
(95-135th) (ee is literally just "| clip") <-- # leave out for custom pipe (fail)
# Else
set dd=%bool:~95,40% <-- "" ""
# "Else" modified for non-string buffers
set dd0=%bool:~96,39% <-- " "
# and copy with ee.. (within is paste as failure pipe)
set "ee=%bool:~-7%" <-- " "
<# %bool% --> #>echo "" | find /I "needle" >nul && (echo Great Success! & ()| clip) || (set "buffer=about:blank" & echo This element does not exist! & echo %buffer%| clip)
# echo "...
set aif=%bool:~0,6%
# ..." | find /I "...
set ahas=%bool:~6,13%
# ... | find /I "...
set "ahas0=%bool:~7,12%"
# ..." >nul && (echo Great Success! & ...
set athen=%bool:~25,33%
# ...| clip) || (set "...
set "cc=%bool:~60,17%"
# ...) || (set "...
set "cc0=%bool:~66,11%"
# ...| clip) || (set ...
set "cc1=%bool:~60,16%"
# ...) || (set ...
set "cc2=%bool:~66,10%"
# ..." & echo This element does not exist! & ...
set dd=%bool:~95,40%
# ... & echo This element does not exist! & ...
set "dd0=%bool:~96,39%"
# ...| clip)
set "ee=%bool:~-7%"
<# %bool% --> #>echo "" | find /I "needle" >nul && (echo Great Success! & ()| clip) || (set "buffer=about:blank" & echo This element does not exist! & echo %buffer%| clip)
# echo "...
%aif%this thing...%ahas%this specific text...
# ..." | find /I "...
echo "this specific element%ahas0% "<------ahas0 completes a quote, too
# ... | find /I "...
echo "this specific element"%ahas%
# ..." >nul && (echo Great Success! & ...
...%ahas/ahas0%the text im searching!!!%athen%
# ...| clip) || (set "...
...%athen%(echo "I've found it!")%cc% #(clips it)
# ...) || (set "...
...%athen%(echo "I've found it!")| clip%cc0% # or, clip it manually / allows to run some loop
#
#
#
#
# ..." & echo This element does not exist! & ...
...%cc/cc0%myerrorvar=true%dd%(echo "my clipboard failure line (#1)"&echo "line2"...))
# ... & echo This element does not exist! & ...
...%c1/cc2%/a count+=1%dd0%(echo my special counter can be found typing "%p%count%p%"))
# ...| clip) <---literal clip pipe, optional:
...%cc/cc0%myerrorvar=true%dd%(echo "my clipboard failure line (#1)"&echo "line2"...)%ee%
We must truly understand those nested echo rules..
b can be nested echo "" "" See:
""<start ---- end>""
But what about c?
c can be nested echo " " See:
It behaves THE OPPOSITE. This will be echoed as a single oneliner block.This compresses html to single line string.
now it makes sense... understanding "d is almost always perfect, cant delimit" ...
There is nothing to delimit...
So now... ahas and ahas0 have clear purpose!
c must use %aif%-INNER-%ahas% :
%aif%(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%)%ahas%sometext%athen%echo trueclip%cc%buffer=bufferran%dd%(echo falseclip)%ee%
INNER-%ahas0% :
# B:
(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)%ahas0%sometext%athen%echo trueclip%cc%buffer=bufferran%dd%(echo falseclip)%ee%
# D:
(echo %0ada%&echo %0adb%&echo %0adc%&echo %0add%)%ahas0%sometext%athen%echo trueclip%cc%buffer=bufferran%dd%(echo falseclip)%ee%
at a glance
With nested echos for b/d, there is one key "weirdness" we solve :
(echo (echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%))%ahas0%aaa%athen%echo trueclip%cc%buffer=bufferran%dd%(echo falseclip)%ee%
therefore, it is cannon for b/d to cut out that first echo
INNER-%ahas0%
So what are the final commands?
Just one.
Search database for any specific (all lines)
You must use c for input. b for output.Original Example
echo "text to search" | find /I "needle" >nul && (echo Great Success! & (echo ["| <a href="https://example.com" title="this is sparta, no, this is a generic T icon"><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACpUlEQVR4nGxTz0tVQRg937z79Hnx0SLNcvEMXBRFq4x+UEKE9EsXbWoR/XLxKqhFEfUHtMpthSiBYUG0kqiNKBFqmBBBWtSiLF9l+CsR7Jm+e+frzL2KL2gud+bemXPOfN+Z+TywKSASDewuYh+Hc/yp52RGA/ 4tIMfvPqxBh9xGfzFHsNy0Cb7NoMWIZJE0SVgDFBRakYFsPQhMjADvBwrw0I4crstT5B1PnNLEKfjr0uiSMjRw2uoiLEIkUODi5fuwu8643QK5e8jgY7dRJHom58JjVQ+Q91wYYTluiU/yHJaQTnvY0ehJ9TZVy/j5mqFO1Z2nPWw5bDHUvSTrpaEyJAe4JHoeu5GUPoYrqN4k9uqAkfKKyBCJc41GCRZVWxtFhnstUi5wVUZY71lB1njMrIAAGhjz9ZXq4EORzfuBPc2QzrOA58P+GhXz7rmilHpWLZLwGFyWntH1gGoejP78LHqnSbT3MZCfgVJZh7uA35OQk/egR66J5imQSBh6pI7rBDK0TKKXeClJQvwEd03FodfyVC88cU4BP0YgxMDaGE+ut5Kj6+KcNQYzyuikZ74AH3qhn17AvOwGKikehMskwBCfi/pVympjZkhXMSwDGXsDLTdMn2Y7vIn2yhli+nnijmeNm1LVmL0stXYjpCwNzH6DGKvGrTMJx3FcQ1I7DQmijFB8ehon9JYmTjONuhPKK+0M1AhLjuMaacOgDdGGFDXVCYmFg2jI3sAcvanYfhyYnyKZOyurg1jHcVzjrvLUPG7oH/TARwkoLSkKvX5E0rTqgSuis98VI88cESgNSxw24hR7Np6FvyGJFn6ymPgsMIGqWqCmDjLKApwdd5YXLMOeYDFVrxTT/8qZoGaTkL283jWuoCg3xgrt5/l3SOu/5fwXAAD//5rxYZkAAAAGSURBVAMAkjI5uqP0pJcAAAAASUVORK5CYII=" alt="An image on the screen displays a generic 'T', This part is read by accessibilty screen readers" style="width:24px;height:24px;"></a><br><span style="font-size:13px; color:#666;">Figure 1 — Example icon footer</span> | Test | Three |"]&echo ["|---|---|---|"]&echo ["| aaa | bbb | ccc |"]&echo ["| ddd | eee | fff |"]&echo ["#### Holy guacamole"])| clip) || (set "buffer=about:blank" & echo This element does not exist! & echo %buffer%| clip)
set "0aaa=| <a href="https://example.com" title="this is sparta, no, this is a generic T icon"><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACpUlEQVR4nGxTz0tVQRg937z79Hnx0SLNcvEMXBRFq4x+UEKE9EsXbWoR/XLxKqhFEfUHtMpthSiBYUG0kqiNKBFqmBBBWtSiLF9l+CsR7Jm+e+frzL2KL2gud+bemXPOfN+Z+TywKSASDewuYh+Hc/yp52RGA/ 4tIMfvPqxBh9xGfzFHsNy0Cb7NoMWIZJE0SVgDFBRakYFsPQhMjADvBwrw0I4crstT5B1PnNLEKfjr0uiSMjRw2uoiLEIkUODi5fuwu8643QK5e8jgY7dRJHom58JjVQ+Q91wYYTluiU/yHJaQTnvY0ehJ9TZVy/j5mqFO1Z2nPWw5bDHUvSTrpaEyJAe4JHoeu5GUPoYrqN4k9uqAkfKKyBCJc41GCRZVWxtFhnstUi5wVUZY71lB1njMrIAAGhjz9ZXq4EORzfuBPc2QzrOA58P+GhXz7rmilHpWLZLwGFyWntH1gGoejP78LHqnSbT3MZCfgVJZh7uA35OQk/egR66J5imQSBh6pI7rBDK0TKKXeClJQvwEd03FodfyVC88cU4BP0YgxMDaGE+ut5Kj6+KcNQYzyuikZ74AH3qhn17AvOwGKikehMskwBCfi/pVympjZkhXMSwDGXsDLTdMn2Y7vIn2yhli+nnijmeNm1LVmL0stXYjpCwNzH6DGKvGrTMJx3FcQ1I7DQmijFB8ehon9JYmTjONuhPKK+0M1AhLjuMaacOgDdGGFDXVCYmFg2jI3sAcvanYfhyYnyKZOyurg1jHcVzjrvLUPG7oH/TARwkoLSkKvX5E0rTqgSuis98VI88cESgNSxw24hR7Np6FvyGJFn6ymPgsMIGqWqCmDjLKApwdd5YXLMOeYDFVrxTT/8qZoGaTkL283jWuoCg3xgrt5/l3SOu/5fwXAAD//5rxYZkAAAAGSURBVAMAkjI5uqP0pJcAAAAASUVORK5CYII=" alt="An image on the screen displays a generic 'T', This part is read by accessibilty screen readers" style="width:24px;height:24px;"></a><br><span style="font-size:13px; color:#666;">Figure 1 — Example icon footer</span> | Test | Three |"
set "0aab=|---|---|---|"
set "0aac=|-|-|-|-|-|-|"
set "0aad=| aaa | bbb | ccc |"
set "0aaa=| <a href="https://example.com" title="this is sparta, no, this is a generic T icon"><img src="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAABAAAAAQCAYAAAAf8/9hAAACpUlEQVR4nGxTz0tVQRg937z79Hnx0SLNcvEMXBRFq4x+UEKE9EsXbWoR/XLxKqhFEfUHtMpthSiBYUG0kqiNKBFqmBBBWtSiLF9l+CsR7Jm+e+frzL2KL2gud+bemXPOfN+Z+TywKSASDewuYh+Hc/yp52RGA/ 4tIMfvPqxBh9xGfzFHsNy0Cb7NoMWIZJE0SVgDFBRakYFsPQhMjADvBwrw0I4crstT5B1PnNLEKfjr0uiSMjRw2uoiLEIkUODi5fuwu8643QK5e8jgY7dRJHom58JjVQ+Q91wYYTluiU/yHJaQTnvY0ehJ9TZVy/j5mqFO1Z2nPWw5bDHUvSTrpaEyJAe4JHoeu5GUPoYrqN4k9uqAkfKKyBCJc41GCRZVWxtFhnstUi5wVUZY71lB1njMrIAAGhjz9ZXq4EORzfuBPc2QzrOA58P+GhXz7rmilHpWLZLwGFyWntH1gGoejP78LHqnSbT3MZCfgVJZh7uA35OQk/egR66J5imQSBh6pI7rBDK0TKKXeClJQvwEd03FodfyVC88cU4BP0YgxMDaGE+ut5Kj6+KcNQYzyuikZ74AH3qhn17AvOwGKikehMskwBCfi/pVympjZkhXMSwDGXsDLTdMn2Y7vIn2yhli+nnijmeNm1LVmL0stXYjpCwNzH6DGKvGrTMJx3FcQ1I7DQmijFB8ehon9JYmTjONuhPKK+0M1AhLjuMaacOgDdGGFDXVCYmFg2jI3sAcvanYfhyYnyKZOyurg1jHcVzjrvLUPG7oH/TARwkoLSkKvX5E0rTqgSuis98VI88cESgNSxw24hR7Np6FvyGJFn6ymPgsMIGqWqCmDjLKApwdd5YXLMOeYDFVrxTT/8qZoGaTkL283jWuoCg3xgrt5/l3SOu/5fwXAAD//5rxYZkAAAAGSURBVAMAkjI5uqP0pJcAAAAASUVORK5CYII=" alt="An image on the screen displays a generic 'T', This part is read by accessibilty screen readers" style="width:24px;height:24px;"></a><br><span style="font-size:13px; color:#666;">Figure 1 — Example icon footer</span> | Test | Three |"&set "0aab=|---|---|---|"&set "0aac=|-|-|-|-|-|-|"&set "0aad=| aaa | bbb | ccc |"
set 0aba=%x:~0,2%%0aaa%%x:~2%&set 0abb=%x:~0,2%%0aab%%x:~2%&set 0abc=%x:~0,2%%0aac%%x:~2%&set 0abd=%x:~0,2%%0aad%%x:~2%&set "0aca=%y:~0,2%%0aaa%%y:~2%"&set "0acb=%y:~0,2%%0aab%%y:~2%"&set "0acc=%y:~0,2%%0aac%%y:~2%"&set "0acd=%y:~0,2%%0aad%%y:~2%"&set 0ada=%z:~0,3%%0aaa%%z:~3%&set 0adb=%z:~0,3%%0aab%%z:~3%&set 0adc=%z:~0,3%%0aac%%z:~3%&set 0add=%z:~0,3%%0aad%%z:~3%
%aif%(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%)%ahas%needle%athen%(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)%cc%buffer=about:blank%dd%echo %buffer%%ee%
generic instead of needle):
Example: Basic
what?..
(ee is literally just "| clip")
Perform no clips:
Example: HTML in input
Set your failure counter:
Searches for "needle" in database (spam this command, paste over and over)%aif%(echo %0aca%&echo %0acb%&echo %0acc%&echo %0acd%)%ahas%needle%athen%echo true%cc1%/a count+=1 & echo ..%dd0%(echo ⇡ my precious failure counter))
%ee% for clip)
Example: HTML in output
Always false:
%aif%the quick brown fox%ahas%a dog%athen%(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)%cc%buffer=about:blank%dd%echo %buffer%%ee%
%aif%the quick brown fox%ahas%wolf|fox%athen%(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)%cc%buffer=about:blank%dd%echo %buffer%%ee%
echo off | clip btw :
%aif%the quick brown fox%ahas%a dog%athen%(echo %0aba%&echo %0abb%&echo %0abc%&echo %0abd%)%cc%buffer=about:blank%dd%echo off%ee%
That's it.
%aif% #echo
(c block)
%ahas% #find
____
%athen%
(echo trueclip) #if html, must be 'b block'
%cc% #set (if else)
buffer=error
%dd% #'else'
(echo falseclip) #if html, must be 'b block'
%ee% #clip
Neat tricks (click to expand)
Have a whole html file you'd like to set?
GRAB B # Replace "set with set | Replace " with nothing
set count=0 && ((for /f "tokens=2" %A in ('findstr /R "^.*" myfile.html') do @set /a count+=1 && @echo -0=["%A"]) | for /f "delims=" %B in ('more') do (@echo "set aa%B"&@echo "set bb%B")) | findstr /R "^.*" | clip
# Replace "set with set | Replace " with nothing
set count=0 && ((for /f "tokens=2" %A in ('findstr /R "^.*" myfile.html') do @set /a count+=1 && @echo -0=["'%A'"]) | for /f "delims=" %B in ('more') do (@echo "set aa%B"&@echo "set bb%B")) | findstr /R "^.*" | clip
# Replace "-0 with -0 instead of set | Replace" " with nothing
set count=0 && ((for /f "tokens=2" %A in ('findstr /R "^.*" myfile.html') do @set /a count+=1 && @echo "-0=['%A']") | for /f "delims=" %B in ('more') do (@echo set "aa%B"&@echo set "bb%B")) | findstr /R "^.*" | clip
Where I left off in this project
Error handling...
b=[""]
c=['']
d=["''"]
str(1) set "a=blank"
realstr(2) set a="blank"
literal(4) set a='test'
actual(5) set a=blank
realrealstr(3) set "a="blank""
# Any failures mean you just need another variant appended. Grab the line you need:
# c and b succeed together on (1) and (3) concurrently.
# To tell c and b apart, c will succeed & b will fail on (2)
# Modify the replacements (these are defaults):
# B: "Replace "set with set | Replace ` " ` with nothing"
# C: "Replace "-0 with -0 instead of set | Replace `" " ` with nothing"
# D: "Replace "set with set | Replace ` " ` with nothing"
Why is this even cool?
"The Snipping Tool will now scan for text and extract it, which can now copy. You can copy all text using the option visible or you can specifically copy part of text from the image, by selecting it."
— Microsoft Learnlink
Win+Shift+T , alternatively, with Microsoft Powertoys
Related projects (on this site):¶
Overview¶
What this page covers
This page serves as the index for the XML Project, documenting literal-safe data handling in cmd.exe:
- The Problem — cmd.exe special characters and parser behavior
- The Solution — Layered encoding system: Base64, delayed expansion, FOR/F, findstr, clipboard
- Media Embedding — Base64 data URIs for HTML elements
- Safety Comparison — Unsafe vs delayed expansion vs Base64 approaches
For detailed component documentation, see the sections below.
The Problem¶
cmd.exe interprets certain characters as control operators at parse time:
flowchart TD
subgraph Special["cmd.exe Special Characters"]
direction LR
AMP["&<br/>Command separator"]
PIPE["|<br/>Pipe"]
LT["<<<br/>Input redirect"]
GT["><br/>Output redirect"]
CARET["^<br/>Escape"]
PCT["%<br/>Variable expansion"]
QUOT["“<br/>Quote toggle"]
end
subgraph Effect["Effect"]
direction TB
E1["Parser interprets<br/>at parse time"]
E2["NOT string delimiters"]
E3["Quote toggles state flag"]
end
Special --> Effect
| Character | Behavior |
|---|---|
& |
Command separator |
\| |
Pipe |
< |
Input redirect |
> |
Output redirect |
^ |
Escape character |
% |
Variable expansion |
" |
Quote-state toggle |
The double-quote character in cmd.exe is not a string delimiter. It toggles the parser's quote-state flag. A " at the end of a line may leave the parser in quoted mode for subsequent lines within a parenthesized block. This is why "delims=^"" requires careful construction.
The Solution: Literal-Safe Encoding¶
flowchart TD
DATA["Raw Data<br/>(contains & | < > ^ %)"] --> B64["1. Base64 Encode<br/>(binary-safe text)"]
B64 --> TRANSPORT["2. Transport Through<br/>cmd.exe pipeline"]
TRANSPORT --> DELAY["3. Delayed Expansion<br/>(!var! syntax)"]
DELAY --> FORF["4. FOR /F Parsing<br/>(structured extraction)"]
FORF --> FINDSTR["5. findstr Filtering<br/>(pattern selection)"]
FINDSTR --> CLIP["6. Clipboard Integration<br/>(clip / Get-Clipboard)"]
CLIP --> DECODE["7. Base64 Decode<br/>(recover original data)"]
style B64 fill:#4a90d9
style DELAY fill:#7ed321
style DECODE fill:#4a90d9
The sHEL approach uses a layered system:
- Base64 encoding — Binary-safe transport of any data through text pipelines
- Delayed expansion —
cmd /v /cwith!var!syntax for safe variable handling - FOR /F parsing — Structured data extraction from command output
- findstr filtering — Pattern matching for data selection
- Clipboard integration — Bidirectional data transfer via
clipandGet-Clipboard
Sections¶
- Base64 Encoding for Media Embedding — Base64 encoding/decoding, HTML data URIs, media embedding
- cmd.exe Literacy — FOR/F, findstr, delayed expansion, parser phases
- Clipboard Automation — clip, macros, tee pattern, file operations
- Haiku Numbersystem Variable Rules — Variable naming conventions for literal-safe scripts
- Countku — Constraint-based language game using English syllable counting
- CAPTCHA Unicode — Invisible Unicode steganography for anti-bot systems
- F# Zensical — Architecture documentation for the F# Zensical documentation system
Tabular Echo: Base64 in Media Elements¶
The following table shows the exact echo command patterns for embedding Base64-encoded media in HTML, accounting for cmd.exe special character handling:
| Media Type | MIME Type | HTML Element | src Format |
|---|---|---|---|
| PNG image | image/png |
<img> |
data:image/png;base64,... |
| JPEG image | image/jpeg |
<img> |
data:image/jpeg;base64,... |
| SVG image | image/svg+xml |
<img> |
data:image/svg+xml;base64,... |
| MP3 audio | audio/mpeg |
<audio> |
data:audio/mpeg;base64,... |
| WAV audio | audio/wav |
<audio> |
data:audio/wav;base64,... |
| OGG audio | audio/ogg |
<audio> |
data:audio/ogg;base64,... |
| MP4 video | video/mp4 |
<video> |
data:video/mp4;base64,... |
| WebM video | video/webm |
<video> |
data:video/webm;base64,... |
| ICO icon | image/x-icon |
<link rel="icon"> |
data:image/x-icon;base64,... |
SVG to Base64 Efficiency¶
SVG (Scalable Vector Graphics) encoded as Base64 is more space-efficient than raster equivalents for geometric imagery. SVG source is XML text, which compresses well and has no binary overhead. The recommended workflow:
- Optimize SVG with
svgo(remove metadata, simplify paths) - Minify to single-line XML
- Encode as Base64
- Embed as
data:image/svg+xml;base64,...
# Optimize SVG
svgo input.svg -o output.svg --pretty=false --multipass
# Encode as Base64 (Linux/macOS)
base64 -i output.svg -o output.svg.b64
# Encode as Base64 (PowerShell)
[Convert]::ToBase64String([IO.File]::ReadAllBytes("output.svg"))
Compared to PNG for the same icon:
| Format | Original | Base64 | Overhead |
|---|---|---|---|
| SVG (simple icon) | 500 bytes | ~670 bytes | 34% |
| PNG (same icon) | 2,000 bytes | ~2,670 bytes | 34% |
SVG maintains crisp scaling at any resolution; raster images require higher resolution (and larger file size) for equivalent visual quality.
Quick Example: Safe vs Unsafe¶
Related Deep Hole¶
- RFC 4648: Base64 Data Encodings — Base64 specification
- SS64: EnableDelayedExpansion — Delayed expansion behavior
- SuperUser: How does delayed expansion work — Parse phase documentation
- Microsoft Docs: setlocal — Official setlocal documentation