File: simple-tokenize.h

package info (click to toggle)
llama.cpp 7593%2Bdfsg-3
  • links: PTS, VCS
  • area: main
  • in suites: sid
  • size: 71,012 kB
  • sloc: cpp: 329,391; ansic: 48,249; python: 32,103; lisp: 10,053; sh: 6,070; objc: 1,349; javascript: 924; xml: 384; makefile: 233
file content (6 lines) | stat: -rw-r--r-- 114 bytes parent folder | download 
1
2
3
4
5
6
 
#pragma once

#include <string>
#include <vector>

std::vector<std::string> simple_tokenize(const std::string &);