Merge pull request #294 from Ambrevar/v2TOC2Renderer

v2: Move TOC generation to the HTML Renderer

Merge pull request #287 from Ambrevar/v2walkfix

v2: Fix walk so that it does not step outside the root node

Fix typo

Merge pull request #286 from russross/add-benchmark

Repeat a run of reference tests under benchmark

Unpublish and rename LinkType constants (#285)

* Unpublish and rename LinkType constants

The constants are only used in the parsing phase, they are not recorded
in the AST directly, so make them private. Improve their names along the
way. Fix tagLength to return two values instead of taking an output
parameter.

* autoLinkType -> autolinkType

And remove unnecessary comment.

Merge pull request #284 from russross/fix-lint

Fix most of lint errors on v2

Return *Latex from NewLatexRenderer

Limit Node.String() to print itself, no children

Merge pull request #274 from Ambrevar/v2misc

v2: Implicit interface and Stringer

Merge pull request #282 from russross/v2-fix-279

v2: Fix issue in fenced code block pre-processing.

Remove dead fields from HTMLRenderer

Add RenderNode to Renderer interface

Remove dead HTMLWriter code

Untangle some mess with attribute escaping

1. Remove unused preserveEntities parameters
2. Move attrEscape() implementation inside esc()
3. Delegate most of esc() work to escCode()

Merge pull request #275 from Ambrevar/v2css

v2: Fix empty path to CSS

Merge pull request #276 from russross/v2-move-test-helpers-to-test-pkg

v2: Only include test helpers inside test package.

Move a couple helpers to parser where they're used

Unduplicate attrEscape funcs

Rename HTML to HTMLRenderer

Move html entity regexp to where it's used

And unify regexp variable names.

Clean up Renderer interface: remove all callbacks

We now expose the structure of the document in the form of AST, there's
no longer need to have all those callbacks to represent structure.

Convert uint32 fields to ints

The former is a bit too inconvenient despite being more accurate.

Get rid of unneeded method in Renderer

Make HTMLRendererParameters a substruct of HTML

Go style: take advantage of zero value init

Cleanup renderer constructors

Move all parameters under HTMLRendererParameters struct, rename
constructors to New{HTML|Latex}Renderer.

Remove a bit of cruft from HTML renderer

Make Markdown the most general API call

Get rid of MarkdownOptions.

Implement SkipHTML, add test

Implement SkipStyle, add test

Fix a bug in findHtmlTagPos introduced with e02c392d.

Implement SkipImages and add test

Allow NodeVisitor to have some control over traversal

Make NodeVisitor return status and decide upon it which node to go to
next. So far, this allows to skip subtrees and quit early.

Give node visitor callback type a name and docs

Fixup broken test reporting

Dereference the pointer introduced in f35fae8.

Add test for UseXHTML

Add test for CompletePage flag

The implementation for CompletePage was committed with d7f1878 by
accident, following up with the test.

Implement SkipLinks, add test

Get rid of unnecessary test helpers

Extract panic recovery code

Call recover() in a single place so it could be easily turned off when
not needed.

Do away with doTestsInlineParam

What used to be doTestsBlockWithRunner is now renamed to
doTestsWithRunner and is good for both block and inline tests.

Fix old omission in doTestsBlockWithRunner

It calls the provided runner in the -short run, but the standard runner
in the complete run. Fix that.

Use TestParams in block tests

The different testing function interfaces get unified a bit closer,
runnerWithRendererParameters completely eliminated.

Wrap inline test parameters in a struct

Take advantage of zero value initialization.

Move all testing helper funcs in one place

Moved verbatim except for uncommented panic recovery section in the
doTestsInlineParam function.

Implement TOC and OmitContents

Move these two flags from HTML renderer's flags to extensions. Implement
both since they were not yet implemented in the AST rewrite. Add tests.

Note: the expected test strings differ very slightly from v1. The HTML
produced by v2 has a few extra newlines compared to the old one, but
it's now uniform with other sections of the generated document. If the
newline placement gets cleaned up in the future, this will get fixed
automatically, since the renderer is agnostic about the TOC list.

Go style: more Html -> HTML renames

Go style: Html{Block|Span} -> HTML{Block|Span}

Go style: rename Toc to TOC

Make ForEachNode func a Walk method on Node

Improve documentation for Node struct

Move Header-related fields to a nested struct

Move table cell fields to a nested struct

Store cell alignment in own type instead of int

Move code block fields to a nested struct

Fix typos

Make ListData a nested struct instead of pointer

Rename HtmlFlags to HTMLFlags to adhere to Go style

Remove dead code

Run Smartypants as a separate pass over the AST

Separate Smartypants somewhat from the HTML renderer. Move its flags
from HtmlFlags to Extensions (probably should be moved to its own set of
flags, but not now). With that done, do a separate walk of the tree and
either run Smartypants processor if it's enabled, or simply escape text
nodes.

Combine two Smartypants structs into one

Combine smartypantsRenderer and smartypantsData into one struct. Make
action funcs methods on that struct.

Rename Html to HTML to adhere to Go style

Make renderer write to an explicit io.Writer

Expose Parse func that grabs input and returns AST

Remove unneeded reference to Renderer from parser

Make common flags and extensions public

Add DefaultOptions convenience variable.

Remove some cruft

The single node renderer is a separate func now

A default HTML renderer for a single node is now easily accessible.
Makes it easy to fall back to the default behavior when writing custom
HTML renderers.

Extract local funcs/vars into methods/members

Fix TitleBlock test

It seems like the implementation of the extension was rather sloppy and
didn't adhere to the style common to the rest of Blackfriday.

Fix footnote tests

This commit does some changes to the test suite. The changes are only to
the newlines. Turns out the former implementation of footnotes produced
slightly different spacing when rendering the footnotes list. The new
implementation produces the list that is compatible with the rest of the
package.

Enable full AST construction, use the new renderer

Connect the block and inline parsers. Most of the tests now pass, only a
couple fail, they need fixes in the test suite.

Construct AST WIP: add inline-level nodes

Parse the content of the block nodes for inline markdown, decorate the
tree with what's found. Tests still broken, need to connect the dots.

Construct AST WIP: add block-level nodes

Build a partial tree by adding block nodes. The block nodes will then be
traversed and inline markdown parsed inside each of them. Tests are
broken at this point until the full tree is constructed.

Add a new renderer from AST

This is the new renderer that walks AST and renders everything to a
buffer. Completely covers all the functionality of the previous renderer
and will likely replace it.

Remove unused code

Add AST data structures

The main Node structure that holds every AST node and some supplementary
substructs. A bit of helper code to handle the nodes.

Fix rendering of tables

Add proper capture groups to fix the order of bits and pieces.

HACK: render TOC the old way, backup and truncate output

Enable writing plain text straight to output

It's only used in a single place and should probably be refactored away,
but this workaround is OK for now.

Eradicate 'out' parameter in top level code

Remove all uses of 'out' in Latex renderer

Remove almost all uses of 'out' in HTML renderer

Redirect output to a capture buffer where necessary

Use CaptureWrites where output should go to a temp buffer instead of the
final output.

Fix Begin/EndHeader to use the new 'out'-less interface

Remove the 'out' parameter. Also, instead of returning and passing the
position of TOC, use CopyWrites to capture contents of the header and
pass that captured buffer instead.

Add infrastructure to collect output in a buffer

Add a structure to collect output in a buffer (replaces what used to be
the 'out' parameter all over the place).

Notable things about this struct are the captureBuff and copyBuff
buffers. They're intended to redirect all the output (captureBuff) or
make a copy of all the output (copyBuff) while they're set to non-nil.
Here's an example of their intended use:

    // what used to be a temp buffer as an 'out' parameter
    //     var cellWork bytes.Buffer
    //     p.inline(&cellWork, data[cellStart:cellEnd])
    // can now be captured like this:
    cellWork := p.r.CaptureWrites(func() {
           p.inline(data[cellStart:cellEnd])
    })

Remove a bunch of 'out' parameters from calls, WIP

Still not all of them, still broken.

Remove 'out' parameter from parser funcs

This only removes the parameter from declarations, everything is broken
at the moment.

Remove 'out' parameter from renderer interface

This only removes the parameter from func declarations, not from their
bodies, so obviously breaks everything. Will be restored in upcoming
commits.

Fix smartypants smart dash processing

Change the way maybeLineBreak gets called to avoid breaking up stretches
of unprocessed characters that smartypants expects.

This inline processing is getting a bit out of hand, something needs to
be done about it.

Remove last call to Truncate() from parser

Autolink detection used to be triggered by a colon and preceding
protocol name used to be rewound. Now instead of doing that, trigger
autolink processing on [hmfHMF] and see if it looks like a link.

Remove calls to Truncate() from linebreak parser

Replace output truncation with appropriate inline callbacks. lineBreak()
is now only responsible for handling HardLineBreak. BackslashLineBreak
is handled in escape() and trailing whitespace is considered in
maybeLineBreak().

Remove a couple calls to Truncate() from parser

Link parser used to truncate in two cases: when parsing image links and
inline footnotes. In order to avoid this truncation, introduce a
separate callback for each of these cases and avoid writing extra
characters instead of truncating them after the fact.

Rename Renderer method receivers

From 'options' to 'r'. This change contains only a massive rename, no
other changes.

Remove callback from Footnotes renderer event

Split Footnotes into two events: BeginFootnotes and EndFootnotes,
removing the need for callback.

Remove callback from Paragraph renderer event

Split Paragraph into two events: BeginParagraph and EndParagraph,
removing the need for callback.

Remove callback from List renderer event

Split List into two events: BeginList and EndList, removing the need for
callback.

Remove callback from Header renderer event

Split Header into two events: BeginHeader and EndHeader, removing the
need for callback.

Simplify callbacks in Renderer interface

The callbacks used to return bools, but none of the actual
implementations return false, always true. So in order to make further
refactorings simpler, make the interface reflect the inner workings: no
more return values, no more conditionals.

Massive replacement of C_STYLE flags to typed ones

Rename public constants to idiomatic Go

Use more idiomatic form for set of strings.

This is a better style for a set, since each value can only be present
or absent.

With bool as value type, each value may be absent, or true or false. It
also uses slightly more memory.

Add a few missing HTML5 block tags

Closes #197.

Sort block tag list alphabetically

Merge pull request #216 from russross/issue-194

Fix footnote following an exclamation point

Merge pull request #215 from russross/issue-193

Fix nested footnotes

Add <style> tag to a list of recognized block tags

Closes #182.

Merge pull request #211 from russross/issue-122

Fix issue 122: fenced code blocks inside blockquotes

Merge pull request #212 from nitoyon/readme

README: Add 'Definition lists' and 'Footnotes'

Simplify return value

Fix broken test and add test for HTML_SMARTYPANTS_DASHES

Merge pull request #190 from anthonyfok/add-flag-for-smart-dashes

Add HTML_SMARTYPANTS_DASHES for toggling smart dashes

Merge pull request #203 from russross/issue-136

Handle comments within a block

Merge pull request #206 from russross/issue-164

Fix footnote followed by a reference style link

Merge pull request #205 from russross/issue-156

Fix bug parsing emphasis

Merge pull request #208 from russross/issue-18

Fix escaping asterisks within emphasis

Bump const VERSION to 1.4

This is probably not used by too many people, but there it goes anyway.

Closes #119.

Add Go 1.5 to Travis config

LaTeX backend: add # to list of characters requiring backslash-escaping

Merge pull request #175 from vbatoufflet/deflist-fix

Fix definition lists end-of-list detection

Merge pull request #176 from tw4452852/index_out_of_range

fix index out of range panic in reference link

Merge pull request #171 from vbatoufflet/deflist-support

Add definition lists extension support

Merge pull request #170 from russross/no-newline-after-img

Do not emit newline after <img> tag.

README: Update import path of github_flavored_markdown package.

It has moved out into a standalone repo recently. See shurcooL/go#19 for rationale.

Fix a couple typos

Merge branch 'jtolds-master'

Merge pull request #161 from rtfb/issue-146

Issue 146

Merge pull request #163 from neclepsio/master

Implement backslash hard line break extension

Add Go 1.4 to Travis config

Simplify isRelativeLink() a bit

Merge pull request #152 from elian0211/about_links

update about links

Merge pull request #159 from rtfb/sequential-footnotes

Fix recognition of consecutive footnotes

Merge pull request #157 from dchest/reflib

Renaming of upskirt

Merge pull request #155 from beyang/master

add rel="noreferrer" option

Merge pull request #149 from tw4452852/fenced_code

Delete unnecessary copy of input when enable fenced code extension

Merge pull request #145 from eparis/license

Move the license info from README to LICENSE file

Merge pull request #142 from anthonyfok/avoid-converting-dates-into-fractions

Avoid converting dates into fractions

Merge pull request #140 from russross/update-import-path/sanitized_anchor_name

Update import path of sanitized_anchor_name.

Merge pull request #138 from KenjiTakahashi/emphasis_link

Fix for #19 + some tests for emphasis with links.

Merge pull request #131 from russross/minor-cleanup

Minor cleanup.

Merge pull request #128 from bjornerik/angled-quotes

Add support for angled, double quotes

Merge pull request #130 from shurcooL/master

Fix test case.

Merge pull request #129 from halostatue/header-prefixes-and-suffixes

Allow configurable header ID prefix/suffixes.

Merge pull request #126 from halostatue/generate-unique-header-ids

Prevent header collisions on generation.

Merge pull request #125 from halostatue/auto-header-id

Add a flag to turn on header ID generation.

Merge pull request #124 from halostatue/fix-header-id-toc-rendering

Use supplied header ID for TOC rendering.

Merge pull request #118 from rtfb/test-relative-links

Make sure relative URLs are preserved

Merge pull request #117 from rtfb/remove-sanitization

Remove sanitization

Merge pull request #111 from rtfb/master

Add Travis configuration and a badge

Fixup tests after 67002b0

Merge pull request #109 from tummychow/html5-code-block

Use HTML5 recommended style of language on code blocks

Merge pull request #107 from shurcooL/master

Improve parser to detect LineBreak independently of renderer.

Fix misleading wording in README

Fixes #103.

Merge pull request #98 from cpuguy83/master

Add titleblock support

Merge pull request #95 from anonx/master

Whitelisted th and td to fix #79

Merge pull request #93 from shurcooL/patch-1

Add a reference to github_flavored_markdown in README.

Merge pull request #88 from dimfeld/master

Remove unnecessary HTML_ABSOLUTE_LINKS flag

Merge pull request #86 from dimfeld/master

Add absolute link transformation and footnote enhancements

Merge pull request #82 from dimfeld/master

Sanitize shouldn't filter out URLs without protocol.

Update README.md

link to markdownfmt, add note about LaTeX output

Merge pull request #78 from JeffPaine/patch-1

Add Usage heading to README

Merge pull request #39 from athom/tilde-escape

allow \~ to escape as ~

Merge pull request #77 from mprobst/cleanup_sanitize_test

Move sanitization tests into their own file.

Merge pull request #73 from johnsto/bugfix/fenced-code-append

Fix fenced code extension modifying data beyond slice

Merge pull request #76 from mprobst/self-closing

feat: Write self-closing tags with a />

Merge pull request #75 from mprobst/sanitize_test

Avoid raw mode parsing so that tags like <script> don't cause escaping

Merge pull request #74 from mprobst/sanitize_test

Add a test for the correct handling of escaped entities in HTML.

Merge pull request #71 from mprobst/master

Add support for a bunch more safe HTML element tags, and bring them into...

Merge pull request #70 from mprobst/master

fix: Handle all different token types that the parser can emit (d'oh).

Merge branch 'gihnius-master'

Allow rel attribute in sanitizer

Fixes issue #68.

Merge pull request #69 from mprobst/master

Use go.net/html's parser to sanitize HTML.

Merge pull request #64 from willnix/master

Add table tags to the whitelist.

Merge pull request #61 from shurcooL/feature/dont-expand-tabs-inside-fenced-code-blocks

Don't expand tabs inside fenced code blocks.

Merge pull request #60 from shurcooL/fix/fenced-code-block-extra-newline

Fix for potential extra newline added inside fenced code blocks.

Merge pull request #59 from johnsto/master

Header ID specifiers

Merge pull request #56 from muhqu/issue/45

Fix for Fenced Code Blocks without a blank line before

Merge pull request #58 from aspic/master

Explicit return byte array at end of function.

Merge pull request #55 from rtfb/master

Autolink fixes

Merge pull request #50 from rtfb/master

Better protection against JavaScript injection

Merge pull request #52 from laslowh/master

add HTML_NOFOLLOW_LINKS

Merge pull request #44 from FreakyDazio/safe-relatives

Relative URIs are considered safe

Merge pull request #43 from microcosm-cc/master

Cells in THEAD > TR are now TH.

Change GOROOT references to GOPATH in README

fix smartypants to pass single backticks through, issue #38

panic fix (issue #33) with test case

Merge pull request #32 from bertzzie/master

Enable Parsing Inside a Link

Merge pull request #31 from aybabtme/patch-1

Fix typo.

update license language to match OSI

Merge pull request #29 from athom/master

add EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK flag to make it closer to GFM

Merge pull request #27 from moshee/master

Footnotes (addresses #14)

Merge pull request #22 from rtfb/master

Add some protection against script injection

Merge pull request #24 from subosito/sample-fixes

Fix table syntax example on README

Merge pull request #16 from cespare/blockcodegithub_doc_fix

Fix up method documentation formatting for the BlockCodeGithub method.

Merge pull request #15 from moshee/master

HTML5

recognize fraction slash as well as regular slash to make fractions

link directly to blackfriday-tool

readme updates for go 1

updates for go 1

version bump to v1.1

permit backslash-escaped vertical bars in tables

fixed bug with blank line handling within list items

table unit tests and fix for a crash uncovered by them

bug in tables when a row has too few columns

tag as version 1.0

additional doc comments

doc improvements, commenting

readme tweak

readme updates

simplify naming of parsing functions

finished removing redundant end-of-buffer checks in block parsing; code cleanup

bounds checking stress tests

missing bounds check

removing more redundant checks, additional cleanup of block parsing

more consistent spacing of block-level elements

remove redundant tests for tab characters in parsing

remove NO_EXPAND_TABS options

removing redundant end-of-buffer checks in block parsing

move whitespace stripping to parser, not renderers

corner case spacing issue with table of contents

simplify inline callback interface

version bump to 0.6

inline helpers put parser arg first

Renderer is now an interface

preparing for switch to rendering interface

table of contents support beefed up

complete page rendering is now an option in the library

refactoring: inline renderers return bools, preparing rendering struct to become an interface

render -> Parser, made parsing functions methods of *Parser

camel case

added simplified interface for common usage

version number, few more options for command-line tool

example markdown binary: try to guess a title

options to supress tab expansion or to expand tabs to 8 spaces instead of 4

fenced code: ending marker must match beginning marker, tests for fenced code blocks

readme tweak

README tweak

simplified BSD license

preformatted html block tests

tests for ordered lists

horizontal rule and list testing

more robust whitespace stripping and matching corrections to tests

fixed minor bugs uncovered by more testing

unit tests for underlined headers, improved whitespace handling for the same

fixed headers nested in lists, added prefix header unit tests

eliminate a buffering level for paragraphs

clean up main markdown function: split out first and second passes

refactoring: newlines as hard breaks changed from HTML option to global markdown option

refactoring paragraph rendering

reduce copying for lists

experiment: render headers directly to output buffer to avoid a copy; minor speed boost

dumb tweak that gives a little speed bump

rewrite of attrEscape: cleaner and faster

enable profiling from command-line tool

fraction example in readme

more inline unit tests

output validates, command-line tool has useful options

improved (hopefully) smart quote handling

convert test files to unix format, fix a few broken ones

unit test for linebreaks

tab expansion bug

fix test name conflicts

Merge pull request #2 from kjk/markdown-tests

integrate tests for markdown 1.0.3 test files by comparing them with refe

starting inline unit tests, fix a few minor bugs they exposed

readme updates

comments, minor cleanups

rewrote the stinking pile of code that was blockParagraph

tab expansion fixed to handle multibyte unicode characters

gofmt

allocate new buffers on stack; mild speed improvement

export all names from Renderer struct
This enables new back-ends that are not part of the package
Basically a big search-and-replace for this commit

performance fix: with autolinking on, it is almost twice as fast now

remove dependency on less function

readme tweak

rudimentary latex backend, additional cleanup

split parser into multiple files, clean up naming

cleanup in markdown: better naming, misc fixes

comments on performance

fix smartypants and html entity escaping

escape entities when using smartypants

return result instead of taking buffer as input

features list

mdash

readme file

refactored into a proper package

smartypants

cleanup

compatibility fixes

output matches upskirt for markdown test suite

fixing link parsing

basics working, still a few renderers to write

parsing done but untested

working on inline parsing

emph parsing

reference extraction

setup, starting reference handling

fixed ordered lists

all block-level parsers

working on listitem

initial commit

integrate tests for markdown 1.0.3 test files by comparing them with reference files rendered with upskirt (no extensions)

Add some HTML5

HTML5 doctype, Wrap TOC with <nav>

<nav> makes the TOC more easily identifiable and workable with CSS.

Fix html tag ordering in doc string.

Fix up method documentation formatting.

fix table syntax example

Improve html element stripping code

Fix typo

Add HTML_SKIP_SCRIPT to MarkdownCommon

Couple more tests

More <script> stripping

Partially addresses issue #11.

Add an option to strip <script> elements

Partially addresses issue #11.

Make isHtmlTag() case insensitive

Extract repetitive code to a func

Fix bug in isHtmlTag()

Fix what seems to be a typo. j should iterate through all tagname, so it
should be initialized to zero. The test exposes this bug.

Make a way to parameterize inline tests

Expose extensions and html flags parameters so that tests could specify
what code paths they want to exercise.

parser no longer returns prematurely from empty footnote ref

leftover debug stuff

added slice bounds check

Implementation and some tests for inline footnotes. Also I noticed the list items had the wrong ids, that was silly of me.

Referenced footnotes appear to be functional. Inline still unimplemented.

new tests pass but old tests now fail...

First attempt at supporting Pandoc-style footnotes. The existing tests have not broken but the new functionality does not work yet.

add testcase for GFM autolink

make autolink peforms like GFM

add EXTENSION_NO_EMPTY_LINE_BEFORE_BLOCK flag to make it closer to GFM(Github flavor Markdown)

Fix typo.

Image inside a link now works.

Added test for link inside image

Updated tests to check for th tags instead of td tags within thead blocks

Added th to table headers so that styling with things like Twitter Bootstrap and typeset.css work as expected. Cells in headers should always be TH unless they are advisory cells within headers in which case TD is acceptable (but being Markdown a user with such needs could just enter HTML for this)

Tests for links when using  HTML_SAFELINK

  Relative URIs are considered safe

add HTML_NOFOLLOW_LINKS

Add a note about JavaScript injections to README

Rename HTML_SKIP_SCRIPT to HTML_SANITIZE_OUTPUT

Rewrite protection against JavaScript injection

This drops the naive approach at <script> tag stripping and resorts to
full sanitization of html. The general idea (and the regexps) is grabbed
from Stack Exchange's PageDown JavaScript Markdown processor[1]. Like in
PageDown, it's implemented as a separate pass over resulting html.

Includes a metric ton (but not all) of test cases from here[2]. Several
are commented out since they don't pass yet.

Stronger (but still incomplete) fix for #11.

[1] http://code.google.com/p/pagedown/wiki/PageDown
[2] https://www.owasp.org/index.php/XSS_Filter_Evasion_Cheat_Sheet

Extract useful code to separate func

Fix bug in autolink with trailing semicolon

In case the link ends with escaped html entity, the semicolon is a part
of the link and should not be interpreted as punctuation.

Fix bug in autolink overescaping html entities

If autolink encounters a link which already has an escaped html entity,
it would escape the ampersand again, producing things like these:
    &  --> &
    " --> "
This commit solves that by first looking for all entity-looking things
in the link and copying those ranges verbatim, only considering the rest
of the string for escaping.
Doesn't seem to have considerable performance impact.
The mailto: links are processed the old way.

Extract a chain of ifs into separate func

This gives a ~10% slowdown of a full test run, which is tolerable.
Switch statement is still slightly slower (~5%). Using map turned out to
be unacceptably slow (~3x slowdown).

go fmt

Fix bug in autolink termination

Detect the end of link when it is immediately followed by an element.

Fix bug with overzealous autolink processing

When the source Markdown contains an anchor tag with URL as link text
(i.e. <a href=...>http://foo.bar</a>), autolink converts that link text
into another anchor tag, which is nonsense. Detect this situation with
regexp and early exit autolink processing.

Explicit return byte array at end of function.

Revert "add an infinity-loop detection to block-level parsing"

This reverts commit 0c62e28e900533ff5d0376fac2e5b0c4894e1fa3.

optimisation: only fix fenced code blocks if the extensions parser flag is set... ;-)

out-comment stderr debug output

add some stderr output to reference stress tests

fix issue #45: 'Fenced Code Blocks without a blank line before'

Add missing newline between paragraph and fenced code block within `firstPass()`.

add error message when panic has been raised within `doTestsBlock()`

add an infinity-loop detection to block-level parsing

add test cases for issue #45

Also support header IDs in ## headers ##

Correctly emit trailing header ID brace

Add Header IDs to default extensions

Add header ID support and tests: # Header {#myid}

Fix bug where newlines were inserted inside fenced code blocks.

Change firstPass() code that checks for fenced code blocks to check all
of them and properly keep track of lastFencedCodeBlockEnd.
This way, it won't misinterpret the end of a fenced code block as a
beginning of a new one.

Add failing test for an issue introduced by PR #56.

The issue is that when there are more than 1 fenced code blocks with a
blank line before and after, the parser introduces a single extra new
line to all the fenced code blocks except the last one.

Don't expand tabs inside fenced code blocks.

Still do normalize newlines inside fenced code blocks.

tagWhitelist allows alignment attribute now

This is the closest I could get to removing everything "unsave" without introducing an additional regex.

Add table tags to the whitelist.

Fixing:
https://github.com/russross/blackfriday/commit/55cd82008e9b35b9a03a80e06d5a4c4601320211

This commit introduced a html tag whitelist which does not include any table tags (<td>,<tr>,<thead>...). Therefore even tables the markdown parser itself generated will be removed.

Use go.net/html's parser to sanitize HTML.

Use an HTML5 compliant parser that interprets HTML as a browser would to parse
the Markdown result and then sanitize based on the result.
Escape unrecognized and disallowed HTML in the result.
Currently works with a hard coded whitelist of safe HTML tags and attributes.

Merge branch 'master' of github.com:gihnius/blackfriday into gihnius-master

Conflicts:
	html.go
	inline_test.go

test: add nofollow ref for non internal links only

add nofollow ref for non internal links only

add target blank attr

fix: Handle all different token types that the parser can emit (d'oh).

Add support for a bunch more safe HTML element tags, and bring them into some order.

Add a test for the correct handling of escaped entities in HTML.

The sanitization code does not retain any particular escaped entities - it
parses the HTML and thus loses the information on what entities were in the
original. The result is correct UTF-8 HTML though.

Avoid raw mode parsing so that raw mode tags like <script> don't cause issues.

Certain tags like <script> but also <title> and others switch an HTML5 parser
into raw mode, which causes the rest of the HTML string to be always parsed as
text, including any elements or entities that we do want to support (e.g. <p>).

As we're going to escape any of the raw text elements anyway (it's e.g. script,
style, title, xmp, noframes, and a couple of others) we can just switch of raw
text parsing by disabling it after each starting tag.

feat: Write self-closing tags with a />

Adds tests for self-closing tags both for correct writing and for correct
sanitization, i.e. stripping attributes on them.

Avoid double alloc

Fix fenced code extn modifying data beyond slice

Move sanitization tests into their own file.

Also adds an explicit test for [link](...) syntax to be sanitized.

allow \~ to escape as ~

Add Usage heading

Sanitize shouldn't filter out URLs without protocol.

Fix spelling error

Add tests for absolute prefix

Add tests for new footnote functionality

Move footnote prefix to a better place

Use parameters object for extra options. Enhance footnote support.

Option to add return links.
Option to make footnote prefixes unique, for rendering multiple
documents per page.

Add footnote prefix option. Needs testing

Add ability to convert relative links to absolute

Remove unnecessary HTML_ABSOLUTE_LINKS flag

Add a reference to github_flavored_markdown in README.

Whitelisted th and td to fix #79

Add titleblock support

Improve parser to detect LineBreak independently of renderer.

When checking if it's a newline preceeded by two spaces, look at the input data rather than the output, since the output depends on the renderer implementation.

Use HTML5 recommended style of language on code blocks

For code blocks that contain a certain language of code, the recommended
attribute structure is <pre><code class="language-foo">. This also
corresponds to the behavior expected by various JS syntax highlighters.

The GitHub code block implementation was obsolete, and identical to the
normal implementation except for its attribute structure, so it was
removed.

Closes #108.

Remove go tip from Travis build matrix

More Travis stuff: better go get, plus build badge

Add a simple Travis config

Change Sanitize() to SanitizeBytes() in example

Document usage of blackfriday along w/ bluemonday

Rip off all blackfriday's html sanitization effort

As per discussion in issue #90.

Convert silly funcs to consts

Make sure relative URLs are preserved

Add tests to make sure we don't break relative URLs again.
Extracted common html flags and common extensions for easy access from
tests.

Closes issue #104, which was fixed as a side effect of cf6bfc9.

Use supplied header ID for TOC rendering.

- Fixes #112 so that `#header {#header-id}` renders the TOC with
  `#header-id` instead of `#toc_1`.

Add a flag to turn on header ID generation.

- Fixes #51, #101, and #102.
- Uses the [code][gfm] mentioned by @shurcooL from his Github
  Flavored Markdown parser extension in a [comment on #102][comment].
  Since this was mentioned, I assumed that @shurcooL would be OK with
  this being included under the licence provided by blackfriday (there
  is no licence comment on his code).
- I’ve added it behind another flag, EXTENSION_AUTO_HEADER_IDS, that
  would need to be turned on for it to work. It works with both prefix
  and underline headers.

[gfm]: https://github.com/shurcooL/go/blob/3bec0366a85101d116bf88b477e92ea6cdd75e4e/github_flavored_markdown/main.go#L90-L102
[comment]: https://github.com/russross/blackfriday/issues/102#issuecomment-51272260

Prevent generated header collisions, less naively.

> This is a rework of an earlier version of this code.

The automatic header ID generation code submitted in #125 has a subtle
bug where it will use the same ID for multiple headers with identical
text. In the case below, all the headers are rendered a `<h1
id="header">Header</h1>`.

  ```markdown
  # Header
  # Header
  # Header
  # Header
  ```

This change is a simple but robust approach that uses an incrementing
counter and pre-checking to prevent header collision. (The above would
be rendered as `header`, `header-1`, `header-2`, and `header-3`.) In
more complex cases, it will append a new counter suffix (`-1`), like so:

  ```markdown
  # Header
  # Header 1
  # Header
  # Header
  ```

This will generate `header`, `header-1`, `header-1-1`, and `header-1-2`.

This code has two additional changes over the prior version:

1.  Rather than reimplementing @shurcooL’s anchor sanitization code, I
    have imported it as from
    `github.com/shurcooL/go/github_flavored_markdown/sanitized_anchor_name`.

2.  The markdown block parser is now only interested in *generating* a
    sanitized anchor name, not with ensuring its uniqueness. That code
    has been moved to the HTML renderer. This means that if the HTML
    renderer is modified to identify all unique headers prior to
    rendering, the hackish nature of the collision detection can be
    eliminated.

Allow configurable header ID prefix/suffixes.

This is specifically driven by the Hugo usecase where multiple documents
are often rendered into the same ultimate HTML page.

When a header ID is written to the output HTML format (either through
`HTML_TOC`, `EXTENSION_HEADER_IDS`, or `EXTENSION_AUTO_HEADER_IDS`), it
is possible that multiple documents will hvae identical header IDs. To
permit validation to pass, it is useful to have a per-document prefix or
suffix (in our case, an MD5 of the content filename, and we will be
using it as a suffix).

That is, two documents (`A` and `B`) that have the same header ID (`#
Reason {#reason}`), will end up having an actual header ID of the form
`#reason-DOCID` (e.g., `#reason-A`, `#reason-B`) with these HTML
parameters.

This is built on top of #126 (more intelligent collision detection for
`EXTENSION_AUTO_HEADER_IDS`).

Fix test case.

Fix a minor issue in expected anchor after recent PR. The tests were written before the improvement that squashes non-alphanumeric characters into a single dash, and does not include dashes at the beginning and end. This updates the test case to match that behavior so that tests pass and Travis is green.

Add support for angled, double quotes

The flag `HTML_SMARTYPANTS_ANGLED_QUOTES` combined with `HTML_USE_SMARTYPANTS` configures rendering of double quotes as angled left and right quotes (« »).

The SmartyPants documentation mentions a special syntax for these, `<<>>`, a syntax neither pretty nor user friendly.

Typical use cases would be either or, or combined, but never in the same document. As an example would be a person from Norway; he has a blog in both English and Norwegian (his native tounge); he would then configure Blackfriday to use angled quotes for the Norwegian section, but keep them as reqular double quotes for the English.

If the flag `HTML_SMARTYPANTS_ANGLED_QUOTES` is not provided, everything works as before this commit.

Minor cleanup.

Apply gofmt on html.go.
Apply goimports-compatible formatting on block.go (space between standard and third party imports).
Move Travis build status image in a more pleasing, common location.
Remove "Markdown pretty-printer output engine" from TODO steps; this is already done in markdownfmt.
Remove unneeded trailing whitespace in README.

fix #19: return immediately from link skipping if end of emphasis is found

add tests for emphasis with links

Update import path of sanitized_anchor_name.

It has moved into a smaller standalone repo.
Closes #139.

Avoid converting dates into fractions

So that dates like 1/2/2005 and 2005/3/4 stay the way it is
without turning into ½/2005 and 2005/¾.

See http://discuss.gohugo.io/t/any-way-to-disable-smart-fractions/328
for discussions.

Move the license info from README to LICENSE file

It's pretty standard to have a license file, and some projects require
them before your code can be used in their code.  Move the license to
its own file.

Delete unnecessary copy of input when enable fenced code extension

Copy of input waste time and memory.

Signed-off-by: Tw <tw19881113@gmail.com>

add rel="noreferrer" option

README: rename libupskirt to Sundown.

The original upskirt library by Natacha Porté has been renamed to
libsoldout, while the fork of it that was located at tanoku/upskirt has
been renamed to Sundown and moved to vmg/sundown. Since Blackfriday has
been started as a translation of tanoku/upskirt, rename its mention in
README to Sundown.

Rename upskirtref to testdata; upskirtref_test.go to ref_test.go.

Fix recognition of consecutive footnotes

The second footnote was treated as if the pair of them were a reference
style link, without checking if the second bit is another footnote.

Fixes issue 158.

update func isSafeLink

update about links
when link to current directory or parent directory

Added test for backslash line break

Add EXTENSION_BACKSLASH_LINE_BREAK

Use EXTENSION_BACKSLASH_LINE_BREAK

Fix previous commit

Backslash was not removed from output.

Implement backslash hard line break

See http://spec.commonmark.org/0.18/#example-527

Cleanup a random bunch of repetitive loops

Replace them with helper function calls.

Allow heading to end with \#

The problem was in a loop that skipped the optional closing hashes in a
heading like this:

    ### This is an H3 ###

Now it checks to see if a hash is escaped and if it is, treats it as a
rightmost character of the heading text, like this:

    ### This is an H3 #\##   ==>   ### This is an H3 ##

Fixes issue #146.

Add a missing test case for reference override

Exercise link text override.

Replace snake_case with mixedCase

Merge branch 'master' of https://github.com/jtolds/blackfriday into jtolds-master

Conflicts:
	inline_test.go

support replacing [refid][] syntax link content with alternate content

Add programmable reference overrides

If a user provides a ReferenceOverride function, then reference ids
will be passed to the given ReferenceOverride function first, before
consulting the generated reference table.

The goal here is to enable programmable support for
"WikiWords"-style identifiers or other application-specific
user-generated keywords.

Example, writing documentation:

 The [Frobnosticator][] is a very important class in our codebase.
 While it is used to frobnosticate widgets in general, it can also
 be passed to the [WeeDoodler][] to interesting effect.

This might be solveable with the HTML Renderer relative prefix, but
I didn't see a good way of making a short link to 'Frobnosticator'
relatively without having to write it twice. Maybe
'<Frobnosticator>' should work? Should Autolinks work for relative
links?

In addition, I wanted a little more richness. I plan to support
Godoc links by prefixing references with a '!', like so:

  Check out the [Frobnosticator][] helper function
  [!util.Frobnosticate()][]

The first link links to the Frobnosticator architectural overview
documentation, whereas the second links to Godoc.

Better advice on how to implement this sort of think with
Blackfriday is highly desired.

Do not emit newline after <img> tag.

This changes HTML renderer not to always add a newline character after
<img> tags. This is desirable because <img> tags can be inlined, and
sometimes you want to avoid whitespace on left and right sides. Previous
behavior of always adding a newline would unavoidably create whitespace
after <img> tag.

Update all tests to match new behavior. There are few changes, and
they're completely isolated to inline image tests.

Fixes #169.

Add definition lists extension support

fix index out of range panic in reference link

Add checking before accessing the input data.

Fix #172 and #173 issues.

Signed-off-by: Tw <tw19881113@gmail.com>

Fix definition lists end-of-list detection

Fix escaping asterisks within emphasis

First check for escaped character, *then* look if i-th character is an
emphasis character.

Closes #18.

Fix bug parsing emphasis

Start searching for emphasis character at 0th index instead of 1st.
Fixes a corner case with doubly emphasised code span followed by
another code span on the same line.

Changes interpretation of improperly nested emphasis, hence the change
in TestEmphasisMix().

Closes #156.

Fix footnote followed by a reference style link

When parsing a deferred footnote, we already know it's a footnote from
the '[^' part, so we can use that to hit a proper switch branch
(default) a bit later on.

Closes #164.

Spread out test cases for readability

Handle comments within a block

Added test cases both for inline and block workflows.

Closes #136.

Add HTML_SMARTYPANTS_DASHES for toggling smart dashes

README: Add 'Definition lists' and 'Footnotes'

Drop misleading comment and turn func into method

* The comment is no longer true.
* Other functions similar to terminateBlockquote() are methods, so make
this one a method too.

Merge pull request #214 from russross/issue-122-test-readability

Make test values more readable, add additional test cases.

Fix fenced code processing inside blockquotes

Add a call to fenced code block processor inside the loop that's
responsible for collecting the quoted lines. Grok all the fenced code
block as a part of the quoted text.

Closes #122.

Move complex conditional to a helper func

Refix fenced code blocks w/o preceding blank lines

Change approach at fixing #45: don't patch input markdown at preprocess
pass, instead improve special case detection when parsing paragraphs.

Leave the fenced code block detection in the preprocess pass though,
it's been put to another use since then, to suppress tab expansion
inside code blocks.

Make test values more readable, add additional test cases.

Minor style fix: declare ref first in a loop

Add String() method to reference struct

Fix nested footnotes

This is both nasty and neat at the same time. All the code could handle
nested footnotes just fine, the only place that was not working was the
final loop that printed the list. The loop was in a range form, which
couldn't account for another footnote being inserted while processing
existing ones. Changing the loop to the iterative form solves that.

Closes #193.

Reformat a dustball of ifs into a switch statement

Fix footnote following an exclamation point

Link parser interpreted the sequence "![^foo]" as an image, but if
footnote extension is enabled, it's quite clear that it should be
interpreted as a footnote following something with an exclamation point
at the end.

Closes #194.

v2: Only include test helpers inside test package.

Fix empty path to CSS

Fix issue in fenced code block pre-processing.

Forwardport changes from #280.

Fixes #279.

Return a *HTMLRenderer from NewHTMLRenderer

Interfaces should be satisfied implicitly.

Make Node satisfy the Stringer interface

It allows for printing AST dumps in external code

Shorten method receivers in Smartypants

Fix bullet points in MarkdownCommon docs

More lint: smartypants.go

More lint: block.go

More lint: inline.go

More lint: markdown.go and ripples to other files

Lint node.go

Fix a few lint errors, a.k.a. improve docs

Enable common extensions for benchmark

Also, reset the timer after loading test data.

Repeat a run of reference tests under benchmark

Will help with optimizing later on.

Fix walk so that it does not step outside the root node

When passed a non-container root node, or when the visitor whould return
SkipChildren for the root node, the former algorithm would go on walking
down the rest of the tree beyond the root.

This commit also removes unneeded functions and variables.

Add TOC generation tests

Break lines properly in TOC generation and omit content if required

Move TOC generation to the HTML Renderer