Skip to content

try to fix #2086 #2087

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions excelize.go
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,60 @@ func OpenReader(r io.Reader, opts ...Options) (*File, error) {
return f, err
}

// StreamRead 流式读取,不支持加密的 Excel 2003
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't introduce new exported function, just updated OpenReader function. Encrypted workbooks should be supported. Please also update unit tests for these changes.

func StreamRead(filename string, opts ...Options) (*File, error) {
file, err := os.Open(filepath.Clean(filename))
if err != nil {
return nil, err
}
f, err := openByName(file.Name(), opts...)
if err != nil {
if closeErr := file.Close(); closeErr != nil {
return f, closeErr
}
return f, err
}
f.Path = filename
return f, file.Close()
}

// 不支持 加密的 Excel 2003 格式!
// openByName read data stream from io.Reader and return a populated
// spreadsheet file.
func openByName(name string, opts ...Options) (*File, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any perf data can be shared about these changes? So, we can compare perf before and after changes. For example: CPU, memory usage, disk I/O.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

我在1G内存的pod 中尝试读取10G大小的 Excel 文件,会直接 oom killed,因为 io.ReadAll 会将整个文件加载到内存里面来;

修改后,内存占用稳定在 54M

f := newFile()
f.options = f.getOptions(opts...)
if err := f.checkOpenReaderOptions(); err != nil {
return nil, err
}
zr, err := zip.OpenReader(name)
if err != nil {
if len(f.options.Password) > 0 {
return nil, ErrWorkbookPassword
}
return nil, err
}
file, sheetCount, err := f.ReadZipReader(&zr.Reader)
if err != nil {
return nil, err
}
f.SheetCount = sheetCount
for k, v := range file {
f.Pkg.Store(k, v)
}
if f.CalcChain, err = f.calcChainReader(); err != nil {
return f, err
}
if f.sheetMap, err = f.getSheetMap(); err != nil {
return f, err
}
if f.Styles, err = f.stylesReader(); err != nil {
return f, err
}
f.Theme, err = f.themeReader()
return f, err
}

// getOptions provides a function to parse the optional settings for open
// and reading spreadsheet.
func (f *File) getOptions(opts ...Options) *Options {
Expand Down